Classifying the IMF-Ukraine Relationships: An Intro to an Intro

Alright I’m joining the hype train of machine learning and AI stuff in economics. Five years after it took off in economics and 20 years since it had departed from computer science faculties into real world. I quite a conservative person, who thinks that by adopting latest trends I present myself as a great intellectual, who observes all that fashion changes from a distance. Have to admit though, this time I gave up.

Why now?

So, Why now? Unfortunately, not because things are going that well. All the credits go to the last chapter of my dissertation, which apparently is not great. Despite all my efforts, I figured out that I’m getting results that could be interpreted in favor of any theory you pick.

And the shame goes to… the data points, that show no connection between change in battle intensity and pace of reforms in Ukraine over 2015 – 2019.

That’s what happens in economics (or in applied statistical analysis) all the time but it’s usually not a great thing if you want to publish your results because you cannot make a great story out it. And the problem is that people in social sciences typically like to hear great stories of sort: “How I have dismantled a well-known theory with my fancy regressions”. Since it is not likely to happen with my dissertation, I’ve decided the following: “OK if cannot get any mind-blowing results, let’s learn show something trendy so the commentators will see that I have provided enough effort and will give me bonus points at least for that”.

Why a tutorial?

Well, one reason is solely practical: the older I become, the easier I tend to forget, what I have learned before. For instance, I have built my first web-scraper in 2017 and spent roughly a month on just to… forget about quite soon, because I am not a hardcode data engineer or a programmer, who spends all the time coding in python. Thus, when I needed to make a new web scraper a year later, I I had to start from a square one again! After the very same situation has repeated once again in 2019, I got a little wild and this text is an insurance against me whining and cursing all day long.

Mr. Travolta showing me reading Python code after a yearly coding break

Still though, why on Earth I do need my own tutorial if Google, stackoverflow and all possible tutorials in the web can answer anything? Well, I simply find it better when you have all (or most) things collected in one place so I don’t have to look up different places any time you are looking for little tweaks related to a specific task you need to perform. Thus one goal I want to achieve with that is if – for some crazy reason – I will abandon python and will have do a similar task again two years later, I will have a source I can refer to without any need to jump from one resource to another.

Second big point is of an autodidactic nature. From my experience, ideas and skills stay in one’s head longer once he structures and brings them down on paper. Thus, by writing the text, I want to nail down the python concepts and scripts deeper in my hand to avoid the situations of type: “And what is the syntax of a python loop, again?” Chances are low, but I outweigh them by keeping my hopes high.

Why public?

Because I believe that I can bring a different perspective to the process compared to the tons of exiting tutorials in the web. Many of them are written by applied data engineers or computer science folks. They are great at demonstrating clean code and breaking down complicated problems in simple manageable tasks. The type of tutorials usually a lack of focus on the conceptual problems that underlie your specific task: they just throw a specific problem at without a discussion whether the exercise they propose is useful in the first place.

Yet this is of a primary concern for a researcher. What is the use of super accuracy of recurrent neural networks for me, in case if I want to reach it I have to classify 100k texts to begin with? Even if I had 10 of research assistants, that would be barely manageable. I am going to lay down my thoughts on that and hope that they will be useful for you, readers, as well.

That was enough mumbling. The next post is going to describe the outline of a problem and set the stage for our research task.

Advertisements

On Pillars of Prosperity

Around 2011, Besley and Persson have published a book, where they present their own view on the factors behind evolution of state capacities of nations. I have struggled to understand the mechanics of the theory, because the models they present are full of corner solutions, Kuhn-Tucker stuff and jumps from an equation to equation without complete derivations (left as an exercise for a reader, apparently). Not like this is something new in economics but a little disturbing in any case.

Nonetheless, they have a plenty of reasonable thoughts, and I tend to agree with their considerations in general. Yet after trying to put myself in the shoes of a ruler (a German philosophy approach, right?), I believe I have found a (big?) miss in their theoretical construct. Since I tend to forget things more and more often these days, I found it necessary to write things on (e-)paper.

intel inside

The basic setup of their model features a society divided in two big chunks: a government ruled by elites and an opposition. Both of them struggle for power to extract rents. The stronger elites are constrained in their executive power, they greater is a piece of common (wealth) pie they share with their contenders.

The core part of the model rests on a presumption that elites extract their rents from taxes (whichever they are). The taxation itself, however, requires investments in a state capacity of a country. This seems as a reasonable assumption because if government wants to collect taxes, it needs bureaucracy capable of doing that, right? This feature of their model implies that elites get a stimulus to invest in public goods: to increase efficiency of public administration elites need to provide education to hire professional clerks for tax departments, find statisticians to keep track of things and “produce” software engineers to maintain databases for an efficient control in the future. Thus, it turns out that the greed for greater rents becomes a mechanism, which might lead nations to prosperity. Want to get greater rents? Fine, but please invest something that has public value to begin with.

But there is a little big problem with that. It’s called “Seniorage”.

Meet the seniorage

Technically, all current governments have an ability to increase supply of money at their will as long as government and a central bank exist in the first place. And those who learned macro-1 know by heart that seniorage is just another type of a tax. The tax, which is super easy to execute, and which works anywhere no matter how good your bureaucracy is. Just print money to buy government bonds and redistribute it they way you like.

Seniorage is a risky strategy, of course. Those, who have lived in high-inflation countries might object claiming that this strategy is suicidal because it is going to cause a social unrest. I believe, however, – based on some recent examples I mention below, that it is not as bad as one might think.

Doing a revolt is not an easy business with many prerequisites for a successful outcome. One needs leaders and united opposition to lead mass protest. One requires proper organization of protesting activity. One should be sure there are enough people ready to go risk their lives (or lives of their families) on the streets. Or in short, it is a problem of collective action, which are so hard to solve. Therefore, many ruling elites bet against it.

Some of them seem to win: look at experience of Mugabe in Zimbabwe or Ugo Chavez in Venezuela. The chances seem to be higher, if leaders are able to frame the reasons of the worsening living standards in a proper way. Usually, finding a suitable scapegoat is a good thing to do (United States foreign policy, evil capitalism, Zionism movements… the list is long!)

The bet becomes even better if one assumes that elites have an escape route to a foreign country and access to foreign assets. If we allow elites to save assets in foreign currency, they escape a problem of being taxed themselves by seniorage and the cookbook for irresponsible elites is ready. Here it is: a) finance your rents by seniorage, b) convert part of the rents into foreign assets/foreign currency, c) save some money to suppress opposition, d.i) repeat the steps a) to c) if the riot is not successful, d.ii) take a plane to a Miami beach just before angry people visit you at home.

Of course the price for that is to face a trial or being executed, but… just imagine the possible profits!

Does it mean that the theory of Besley and Persson is useless? Not necessarily. One can imagine leaders, which do not wish to go for seniorage by no means (Putin seems to be like that) or because elites value long-term security more than immediate income gains (Singapore under Lee Kuan Yew and his successors, maybe?) In these cases, the mechanisms Besley and Persson propose seem to be at place.

A big question, however, is: if the whole framework begins to rely on personal traits of elites, why do we need a complex theory with corner solutions for mechanisms, which are just the consequence of that instead of making a theory for value systems of elites?

Setting Stata and Latex to love each other

Hours. So much can it take, to format the Stata output into the Latex convenient format. Manually deleting rows and columns, manually adjusting symbols and hell knows what. Even after writing the full paper I did not learn how to do it with the minimum time losses. It’s discouraging, unproductive, error prone and all that. So, by the start of the second paper, I decided to put it to the end. Thanks to Internet here and there I was able to find my favorite formula with the \threeparttables package, which eliminates all the horrible work with the footnotes. I post the code as a reminder for myself because as my experience has shown, looking through the old code might be very… unpleasant.

esttab ///
$regressions ///
using Tables\regressions.tex, replace ///
cells(b(star fmt(%9.3f)) se(par)) star(* 0.1 ** 0.05 *** 0.01) ///
legend booktabs noobs nonotes nomtitle collabels(none) ///
keep($covars) label ///
varlabels(_cons Constant) mlabels(“Model 1” “Model 2” “Model 3”) ///
prehead(“\begin{table}[!ht] \begin{center} \begin{threeparttable} \caption{Impact of the battle-events on PFTS index estimated by 2-state Markov-switching dynamic regressions.} \begin{tabular}{l c c c}” “\toprule”) ///
posthead(\midrule) prefoot(\midrule) ///
postfoot(“\bottomrule” “\end{tabular}” “\begin{tablenotes}[flushleft]” “\item Standard errors in parenthesis, * p$<$0.1 ** p$<$0.05 *** p$<$0.01” “\item Economic variables are log-transformed weekly averages. The battle events are inverse sine hyperbolic values of the reported weekly values. All variables are differenced with respect to the previous week.” “\item Sources: PFTS by \cite{pfts2018index}, central bank key rate by \citep{nbu2018rate}, UAH/USD exchange rate by \cite{investing2018exrate}, electricity consumption by \cite{uaenergo2018cons}, Dow Jones Index by \cite{fred2018dowjones}, battle events by \cite{zhukov2017introducing}.” “\end{tablenotes}” “\end{threeparttable}” “\end{center}” “\label{table:reg`suffix’}” “\end{table}”)

The most important parts of the command are the \prehead{} and \postfoot{}, which generate the header preamble with the threeparttable-package and the “afterparty” tablenotes in the bottom.

There are still some inefficiencies in the code such as mtitle(x y z), which I can likely automate even more by using the @span command.

War and Economics of Donbass: A Concise Summary

I have recently finished working on the paper dedicated to estimating the costs of conflict in Donbass, Ukraine. After finishing the full-scale paper I decided to make a sci-pop version, which will present the context of events to a new reader. It does not demonstrate the complex part of the work related to the estimation of the impact of the separatist control on the economies, it gives a good grasp of what is going on there.

Here I post only a reduced-form version of the “story”. One can assess the full version at the ArcGIS web-site (a brilliant storytelling tool) using the link: https://arcg.is/19quTr

Intro

Since April 2014, Ukraine is torn apart by the deadliest armed conflict in Europe since the Yugoslav wars. While the war actions have been making the news headlines, two out of six million people living in the region prior to war left their homes to escape violence and insecurity. The other four decided to stay, with roughly a half remaining in the separatist areas. The life within the separatist republics remains largely unknown. The statistical departments of Ukraine do not collect figures there, data of the separatist republics are scarce, the media investigations remain episodic. The column combines the actual satellite view over the separatist-controlled areas with the available GIS sources and photo evidence to figure out how life in the terra incognita of Ukraine does look like.

When Darkness Makes You See More

As of August 2017, 2897 satellites worked day and night flying around our planet. We use them every time when we turn on Google Earth, check our location with GPS or measure length of a track after an evening run. Many of them help us to see Earth when the lights are on. Yet sometimes one can see from outer space more when the lights are off.

The Visible Infrared Imaging Radiometer Suite (VIIRS) allows to track Earth at night at a resolution of 742×742 meters. National oceanic and atmosphere administration of the US captures the images, calibrates them to assure comparability over time, removes clouds, and provides them on their web-site for free. What could be a military secret thirty years ago is now available for everyone with a basic laptop and internet connection.

Recently, economists began to use nighttime luminosity (brightness) as a proxy for economic activity. Although it is an imperfect substitute for common metrics as GDP, it turns out to be useful for regions with poor or absent statistics.

World_at_Night
NASA-compiled nighttime world image of Earth.

One may notice that the lights are unevenly distributed over the space with most of the light located in or around the cities. The pattern is common for virtually any country and reflects how deeply urbanized the contemporary economies are.

The brightest spot on the mid-north of Ukraine is Kiev, the capital of the country. It can also help to analyze data at a different scale. One may stay at a country-level…

Ukraine_Night_Overview2013
Ukraine at night. January 2013.

…examine certain regions…

EastUkraine_Night_Overview2013
East Ukraine at night. January 2013.

…or do a city/grid-level analysis.

S_Donetsk_N14
Donetsk at night. January 2013.

Not all settlements of Donbass, experienced the war with the same intensity. The territorial control of the separatists was not stable over time and initially spread far beyond the borders they are controlling right now.

The armed confrontation between the state and separatist forces began in the city of Sloviansk, when a group of Russian ‘volunteers’ under command of Igor Girkin stormed police headquarters and set up checkpoints in April 2014.

Separatists_Jun14
Separatist-control in East Ukraine by June 18th, 2014.

In an interview, Girkin acknowledged that his actions became the starting point of the war: “It was me, who pulled the trigger. If our squad would not cross the border, everything would end up as in Kharkiv or Odessa: several dozens of killed, injured, and arrested people and that’s all. It was us, who launched the war, which is going on.”

Until May 2014, the state forces were hardly engaged into combat, and separatists could easily expand their territorial control. By June 2014 they controlled the most populated areas of Donetsk and Luhansk regions except for Mariupol.

The state forces started the massive offense at the position of the separatists in June and kicked them out of the initially controlled city of Sloviansk and it’s surroundings. Separatists troops, however, escaped the encirclement and retreated to Donetsk and Luhansk. The skirmishes came to the most urbanized and the most populated areas of Donbass.

Separatists_Jul14
Separatist-control in East Ukraine by July 17th, 2014.

By August 2014, the state forces managed to lock the separatist troops in the cities and restored control over a large part of the state border with Russia. Yet in two weeks, mass reinforcements (estimated of several thousand people) crossed the Russian-Ukrainian border, encircled several military units of Ukraine, and expanded the war front to the south of Donetsk heading for Mariupol. The Ukraine state forces had to retreat from the cities of Donetsk and Luhansk.

Separatists_Sep14
Separatist control in East Ukraine by September 23rd, 2014.

The battle intensity reached its peak in August – September 2014, declined somewhat thereafter and stabilized (but not disappeared) at roughly the same level since then. Separatists were able to expand their control to the North-West of the Russian border but never reached the initial plans to expand farther away to Crimea.

Separatists_May18
Separatist control in East Ukraine by May 10th, 2018.

The last major territorial change occurred in February 2015, when the separatists troops took over the Debaltsevo settlement, which eases communication between the separatist-controlled Donetsk and Luhansk Peoples Republics (DNR and LNR). Although artillery strikes and local skirmishes is a daily business at the contact line, no large changes in the territorial control occurred since then.

The Art of War…

Albeit representatives of the separatists and the government of Ukraine committed to facilitate the peace process by signing the Minsk-truce in 2015, the agreement remained largely on paper. OSCE observers deployed at the contact line report that state and separatist-controlled troops constantly use heavy arms in the war actions.

They include tanks…
…heavy artillery…
…and multi-rocket launch systems (“Grad”).

The latter ones are especially dangerous to apply against an enemy hiding in cities. The machines were designed to damage the whole areas or districts and are therefore highly imprecise. Hitting the target in a city damages local population no less than the enemy covered by the high-rise buildings.

Yet the ‘needs’ of war dominate the humanitarian considerations when arms are loaded, troops are ready for action and an ‘enemy’ is at the firing distance. Breaking the vicious circle of violence proved hard in all armed conflicts. Ukraine did not become an exception.

…and the Consequences.

The art of war made many people flee. Those, who stayed had to deal with the food shortages, artillery strikes, and movement restrictions within DNR and LNR.

The city of Donetsk is the largest urban area in the Donbass region. Prior to war it had around one million inhabitants and hosted branches of the major industry enterprises of Ukraine.

S_Donetsk_N14
Donetsk in January 2014.

No wonder why the city looked like a big light bulb at night of January 2014. But just a year after, everything has changed. Darkness captured the outskirts and suppressed the lights within the core of the city.

S_Donetsk_N15
Donetsk in January 2015.

By 2016, luminosity started to slowly recover.

S_Donetsk_N16
Donetsk in January 2016.

Yet even by January 2017, the city did not reach the pre-war luminosity levels with the outskirts to the West still captured by darkness.

S_Donetsk_N17
Donetsk in January 2017.

The city of Donetsk is not a unique case. Here is what happens when one calculates a difference in the luminosity for every cell in the Donetsk and Luhansk regions between January 2014 and January 2016.

cropped-2017-11-26_20-15-56.png
Change of luminosity between January 2016 and January 2014 in East Ukraine.

Black cells show a decline, while the yellow ones indicate an increase in luminosity. Only cells with values outside of two standard deviations are colored in black or yellow to assure that the changes are not driven by statistical discrepancy.

Albeit the largest non-rebel city of Mariupol emitted less light, the spots with an increase in luminosity are not rare in the state-controlled areas. Even Sloviansk – the city where the armed insurgency began – shows an increase in luminosity compared to the pre-war level. Yet it is hard to say the same for the separatist-controlled areas of Donetsk and Luhansk, where the luminosity decline was almost universal.

There are several reasons why the effect of war is different for the separatist- and state-controlled areas. One may be tempted to conclude that the separatist governments are simply bad economists. The assessment is not completely wrong but it is not the whole story.

Coal and iron ore were the founding pillars of the Donetsk economy. According to the state statistics of Ukraine, mining, energy, and manufacturing accounted for one third of the Donbass total production in 2013. Coal fueled the local manufacturing enterprises as a source of energy for the local thermal power plants and heavy industries that exported their goods to Russia and Europe.

Together with the railway lines, they established a dense network of enterprises with low-cost supply chains. Yet, the advantage turned out to be a trouble when separatists took over the resource base of the region cutting the traditional connections.

A separatist examining the captured coal mine. (Photo by Max Avdeev)

According to the estimations of the Ukrainian authorities, most of the coal mines captured by separatists are either idle or were flooded to prevent the mines from collapsing.

Trade and service firms in DNR and LNR suffered from selective nationalization of assets, an unpredictable tax system, and an insecure business environment. The photo demonstrates a retail shop nationalized by the DNR authorities and framed as “returning” the property (apparently from oligarchs) to the state.

The subtitle says: “DNR: Everything has just began!” (Photo by Elena Gorbachyova).

The other factor is the severe restrictions the local economies are subject to since winter 2014/2015.

The government of Ukraine imposed two round of sanctions on the economies of DNR and LNR, when it became clear that the state forces will not be able to defeat separatists in a military confrontation. Ukrainian authorities withdrew public services, prohibited social security payments, and imposed a ban on banking operations within the separatist-controlled areas since December 2014. In January 2015, the government introduced a second round of restrictions. This time, it prohibited trade with the “occupied” territories and limited the movement of people in and out of the separatists areas to six checkpoints only. The ‘policy’ meant drastic implications for the region, which had a high positive trade balance prior to war and tight connections with consumers and suppliers beyond both sides of the contact line.

Stanytsa Luhanska checkpoint. The checkpoints are overcrowded with people crossing the “contact line” between the separatists and the government territories.

Everyone, who wanted to avoid the restrictions or get social security payments had to re-register himself  – or the business – in the “mainland” Ukraine and start paying taxes there. Yet the “tax pie” is by no means less important for the separatist authorities than it is for the government of Ukraine. Whereas Ukraine’s government used economic punishment as a “motivation”, the separatists used threats and coercion. The story of Andrey Shabanov – the CEO of the Donetsk coke production plant – who was kept in a car trunk for several days with a demand to re-register the enterprise in DNR, is likely to be the tip of the iceberg. Some entrepreneurs are now calling DNR and LNR “Somalia” for how risky it is to do business there.

Despite all the odds, some enterprises were able to continue run their business. Some of them gamed the system(s) by registering firms in both government- and separatist-controlled areas. The others substituted Ukrainian suppliers by the Russian ones. Smuggling and bribery are also valid instruments.

Until February 2017, however, several big exceptions were towering behind the landscape of small men and women in their strive to survive.

The First Ones Among Equals

Companies that produced the goods of “strategic importance” and registered their enterprises in the government-controlled areas could continue to trade with the rest of Ukraine across the contact line. The energy holding DTEK belonging to Rinat Akhmetov – the richest entrepreneur and oligarch of Ukraine – was the most famous example of the first among equals in DNR. Paying taxes to Ukrainian authorities was balanced by keeping the local workers employed. The fact helped to protect the enterprises from a takeover by the separatist authorities and kept a fragile status-quo stable. For some time.

Banner of the Komsomolets Donbassa coal mine, which belongs (belonged?) to the largest energy- and coal-mining enterprise of Ukraine: DTEK (photo by IO.UA).

Not everyone in Ukraine found the art of doing business morally acceptable. In view of some people – primarily the war veterans – companies like DTEK were trading on the blood while young men were dying at the contact line. The debate around the trade became an important topic in public discussions and raised tensions in Ukrainian politics.

In January 2017, a group of activists decided to act and blocked two railway lines on the way to DNR.

Photo of the activists’ group in winter 2017 (photo by political critique).

The actions of the activists were successful. A number of enterprises stopped production in February due to the railway blockade. The response of the separatists was immediate.

DNR leaders accused the government of Ukraine of inaction and introduced and “external administration” at 46 enterprises until the blockade is uplifted. The Ukrainian authorities responded to the demands of the separatists with a delay and made the railway lines free only in March.

Nonetheless, the separatists did not withdraw the external administration. As a response, the government of Ukraine imposed a complete trade ban with the separatist economies. In the end, everyone became equal but worse off.

The only way for the local firms to export their products lies across the Russian border nowadays. Most of the Donbass produced goods have close substitutes in Russia, which is consuming only a chunk of what Donbass may produce. Thus to earn some money, the products should reach customers outside of Russia. Yet achieving them is difficult as neither DNR nor LNR are recognized by any single UN member state including Russia. It means no bank accounts, no financial transactions, and no contracts.

DTEK managers and journalists calculated that separatists transported roughly 2 mio tons of coal in 2017 across the Russian border – five times less than the supply to the “mainland Ukraine” in 2016 before the complete trade ban. According to the estimates, half of it was consumed in Russia and the remaining one exported to the third countries through either the firms registered in Russia or South Ossetia.

The region that was once one of the largest net-exporters of Ukraine, is now largely dependent on direct and indirect subsidies from the Russian government in form of a non-paid gas supply, humanitarian help, and finance of the state budgets.

Epilogue

As diplomatic negotiations remained stalled with no signs of violence to stop, there little is room for optimism. It is likely that Ukraine will enter the elections of 2019 with the war on the shoulders of its citizens. For many of them, life has changed forever. Those who left their homes were lucky if their friends or relatives hosted them. The destiny of others was way worse.

Many of those who moved to camps for internally displaced people in Ukraine are still struggling to get a permanent accommodation. They continue to live in modular houses where roofs are leaking and walls start to fall apart.

Modular houses built for internally displaced people in Ukraine.

Those, who could not or did not want to move away, witnessed the war in all its brutality.

Regardless of whether they stayed at home… (photo by Georgio Bianchi)
…or went to the front-line… (photo by Max Avdeev)
…the war found them with it’s consequences sooner or later (photo by Max Avdeev).
When buying basic goods may easily turn into a trouble (photo by Max Avdeev).
When one may catch a bullet at a workplace (photo by Konstantin Solomatin).
When looting is the least of all crimes (photo by Max Avdeev).
When checking a rifle turns into a morning exercise (photo by Georgio Bianchi).

And while the remaining pieces of coal are getting burnt, it is getting less and less clear what will disappear first: the deadly bullets or the light of life.

Credits

Based on the working paper “Dying Light: War and Trade in the Separatist-Controlled Areas of Ukraine”.

Photos by

Max Avdeev, Georgio Bianchi, Elena Gorbachyova, Konstantin Somolatin, OSCE, UNIAN, BBC, Reuters, Euromaidanpress, io.ua, Kyiv Post, Political Critique

Videos by

The Guardian

Rasters of the separatist control

mediarnbo.org, informnapalm.org

Administrative borders

GADM Project

Nighttime luminosity imagery

Earth Observation Group by NOAA

List of Nationalized Enterprises

Euromaidanpress.ua

Media reports

BBC.com, Euromaidanpress.ua, KyivPost.com, RBC.ru, UNIAN.ua, Forbes.ru, DTEK.com, theGuardian.com, Spektr.Press, Zavtra.ru

Proofreading

Malika Kurbanova

Idea and Text

Artem Kochnev

Digitizing the Renaissance Italy Provinces

Overture

One of the favorite parts of the history lessons I had at school was looking at maps of national borders. Especially in a time sequence, when they changed back and forth showing a beat rate of a nation. Unfortunately, changes in borders make a long term quantitative analysis somewhat complicated. Imagine you get a wonderful dataset on consumption and income level in, say, Renaissance Florence. Brilliant, but to which territory one could “assign” the number you got? If you think one can do it for the whole Italian peninsula, let me upset you: a lot of work by historians before tell us that the income level in the Northern Italy was way greater than in the South. Luckily, nowadays one does not have to be a trained geographer to address this kind of problem. With a piece of modern software, a computer mouse, and a bit of a patience, one can create a digital file from any existing historical map (okay, almost any). Here is how it goes.

Prerequisites

Digitizing maps is like painting: take a tool, make a touch, make a pause a have a deep look (repeat and correct when needed!) Yet, every painter has his own tools and we are no exception. I bet there are many-many-many programs that may do the things we need, but what most people use (and me as well!) are two programs: a laggy-buggy-open source QGIS, and a fancy-commercial-but-stilly-buggy ArcGIS. Both are bad enough to work with sometimes with some pro and contra for either side. In the lesson, I am going to use QGIS because it perfectly suits our purposes and has a bit more intuitive interface for the newbies like me 🙂

After installing and running the first project in QGIS you will need to take care of two things: i) a raster image we are going to process and ii) shapefile of a country/area. We will need the first one to do our job and the second one to have a reference for a sanity check.

In the case, I am going to use the map that shows the city-states of Italy as of 1494. The map itself is likely to be an approximation of what the real state of affairs was back then given that geodesy as a science probably did not exist back then. But as it often happens in life, truth is often somewhere far away and one has nothing else but to use a second-best solution. Yet, there is a quality threshold we cannot move beyond: the raster image should contain the georeference grid and the coordinate system the map is projected into. Luckily,  the map we are looking at has an explicit notion of Mr. Greenwich at the very top of the image. The fact gives us a hope (just a hope!) that the standard WGS84 projection might work.

2018-02-04_11-01-43.png

Unfortunately, there is no other way to figure the used projection except of doing the manual georeferencing of the image unless it is explicitly defined on the image (yet, as my practice shows, the code of good (geographer’s) conduct is not that common for the images you find in internet and old maps).

Nailing down the Grid

For that, go to the Raster menu and select the “Georeference” button. The main (and tedious) job now is to select the points on the image and assign the values in the coordinate system we want to project the image on. In practice, using a grid intersection is an approach that guarantees the best fit. As the grid values show the coordinate system we are working, one does not need to wonder, which value to assign! Just take the latitude and longitude and put the numbers in the right order.

2018-02-04_11-11-51.png

In my case I got 25 grid points (don’t forget to save them – QGIS is ready to crash at any moment!). Technically, I could do more, if I would manually project other grid lines on the image in a given coordinate system. But it requires extra time and some extra work in graphic editors, which is not worth spending on in the case of sample tutorial. This implies, however, that the fit outside of the grid can turn out to be way worse than for the rest of the image. You will be able to see it visually after the reprojection is finished. And the crucial step do it right is to select a proper projection system. In our case, the initial guess was the world-famous WGS84. Let’s try it!

2018-02-04_11-12-12.png

After some intense work, your PC should add a new raster layer to the project. Most likely, the image will be slightly deformed. That’s ok, because Earth is a spheroid, not a plane. And every projection inevitably introduces a mistake. The question is, whether it is going to be a small or a big one.

2018-02-04_11-13-44.png

In this case it looks that the image did not change that much. But if you want to verify your eye-scrolling, the best way is to take an existing shapefile, which is done right and compare how good the shapes of two coincide with each other.

2018-02-04_11-13-35.png

Not as bad, as one might expect, right? The fact that both the shapefile and the reprojected image largely coincide with each other tells that WGS84 was the right choice.

2018-02-04_11-47-03.png

The Painting Time

Now, when we have a ‘proof’ that the map we use is indeed projected correctly, one can start drawing polygons using the raster image as a painting back matter. Let’s say I want to digitize the land in possession of the Renaissance Florence (the orange area on the picture).

To do that in QGIS, go to LayerCreate a new vector layer (do not confuse with import!) – Create a new shapefile. Select the type as polygon and add the attributes you need (I’ve just added a name). After defining a directory of the file, select the new vector layer, enter the editing mode (a pencil icon on the top panel) and start drawing the points on the map following the image contour you need to digitize. Once finished, just click the right mouse and enter fill in the attribute values.

2018-02-04_11-47-15

Now, if you want to have an additional detail to the map – say the cities – you can just add another vector layer following the exactly same algorithm. The only difference will be that instead of the Polygon type of the vector layer, one has to select a Point or a line (in case you want to draw a border, a river or a trade route).

2018-02-04_11-47-24

This is how the final result looks like when digitizing the location of the cities. In the end, you end up with two vector layers: one for cities and the other one for the controlled area. With that, your figures for income levels might get a new life!

***

You can download the input files and the resulting vector files together with the QGIS project file using the following link: https://yadi.sk/d/m_rIMfcG3SA9qF