Monday, May 11, 2015

Unconditioned by the past

Exploring the Memoryless property of the Exponential Distribution. Cross posted over at 3 Quarks Daily.
1. Waiting For the Next Customer

Suppose you run a small business, a barber shop or a small restaurant that takes walk-ins only. A customer has just left, your place is empty, and you are waiting for the next customer to come in. You've figured out that on average the time between two successive arrivals is 15 minutes. However, there is variation and the variation follows the Exponential probability curve shown in the figure below. This is not an arbitrary choice: time between successive random and independent arrivals does actually follow the Exponential. The average time between arrivals depends on whether it is a busy or slow time of the day, but the general shape of the Exponential curve keeps showing up again and again when empirical data is plotted (one example here).    
The height of the curve is an indicator of where the greatest probability densities are. Most arrivals happen in quick succession (the curve is tall when t is small), but there will be occasions when a long time elapses before the next arrival happens. At t=0, when the last customer just left, if you calculated the probability of the next customer arriving within 5 minutes (0 < t < 5) you would get the value 0.283. Equivalently you could say that the probability you will wait 5 minutes or more is (1 - 0.283) = 0.717. 

Now here's the interesting part. Suppose twenty minutes have now passed and the next customer still hasn't arrived. You are starting to get a little impatient; after all you don't want your productive time to be idle. So at t=20, you again calculate the probability of a customer arriving in the next 5 minutes (20 < t < 25), given that no one has come so far. You would think this new probability, based on how much time has elapsed, should be higher than 0.283. But, surprisingly, the probability that a customer will arrive in the next 5 minutes, given that twenty idle minutes have passed, is still 0.283! And the probability that you will wait 5 minutes or more is still 0.717.

This is precisely the Memoryless property of the Exponential: the past has been forgotten; the probability of when the next event will happen remains unconditioned by when the last event happened. Fast forward even more: let's say you've waited for half an hour. No one has shown up so far. Frustrated, you recalculate the probability of someone arriving in the next 5 minutes (30 < t < 35). Still 0.283!

The behavior that we see in the Exponential is because your customers are arriving independently of one another -- remember that you allow only walk-ins. There is no "memory" or predetermined schedule connecting any two successive arrivals (in the same way that the outcome of a coin that is tossed now has no memory or connection with the outcome of a coin tossed at some point of time in the past). A barber shop, a small restaurant, a shoe-shop, a cab-driver, a car mechanic, a self-employed person who earns a living doing Japanese-English translation requests – one can find many contexts that experience the Memoryless property. Bigger retail firms also experience the same problem, but they hire (and fire) many people, cross-train their employees to do multiple tasks and thereby have ways to reduce the risk of staying idle.

In small businesses, the wait for the next customer is felt far more personally and acutely. Recently, I spoke to Abel (not his real name), an Ethiopian man who had started a restaurant in a small Midwestern town. The Ethiopian dishes I tasted were excellent. Yet Abel said there were many difficult evenings he would be alone, waiting for someone to come in. To cut costs, he was both the cook and the server on such slow days. But Abel noted that he would, unexpectedly, get busy. This is the flip side of the Exponential: a string of closely spaced arrivals is very likely since the probability densities are front heavy, as seen in the shape of the curve. So you can go from being idle for an hour to suddenly having a long line of people waiting. Now you have a different problem – you are too busy and your customers are unhappy! 

2. A Visual Illustration

Let's look closer why exactly the Memoryless property holds true for the Exponential. Instead of showing the algebra, I'll try illustrating visually. I struggled with the Memoryless property myself for many years; so at the very least, I'll put my own thoughts in order. Please let me know if something does not sound right.   

The Exponential is a continuous distribution used to characterize the probability of time durations, such as the time between two successive randomly occurring events. Naturally the smallest possible value is 0. The Exponential curve is asymptotic – a fancy word for the idea that the probability curve keeps dipping as we move to the right and gets closer and closer to the x axis, but never quite dips enough to touch the x-axis. The dipping curve stretches to infinity. So very long time between events (long periods of idleness) are theoretically possible, although in practice they are very, very unlikely. The area under the Exponential probability curve, if we calculate the limit, tends to 1 (as it must for any continuous probability distribution).   
Rescale2Let's return to the original example. Time between successive arrivals follows an Exponential Distribution with a mean of 15 minutes. Currently, twenty minutes have passed since the last arrival, so we are at t=20. We are trying to find out the probability that an arrival will happen in the next 5 minutes -- in the interval 20 < t < 25. To do so, we now only need to consider the area under the Exponential curve to the right of the t=20 mark. The total area under the curve to the right of t=20 is 0.263. We "rescale" this area such that 0.263 now becomes equivalent to an area of 1 -- we do this because this is the relevant conditional probability space we are now interested in. Further, the x axis is re-scaled: t=20 becomes t=0; t=25 becomes t=5; and so on, so that in the newly re-scaled or conditioned area we can calculate the probability of an event happening in the next 5 minutes.  

Surprisingly, the re-scaled area is exactly the original Exponential probability curve! Even the height of the curve corresponding to every time value on the x-axis is exactly as it was when the last customer left.

It's not a precise analogy, but just as the same pattern keeps repeating itself in a fractal no matter how much you magnify the original, so the same exact Exponential curve we started with keeps appearing again and again upon re-scaling no matter how much time has elapsed. It does not matter if 10, 30, 100 or 2000 minutes have passed without an arrival; the probability that an arrival will happen in the next five minutes will always be 0.283. This makes mathematical calculations very straightforward -- the past does not need to be kept track of, and the same formulas can be used at any stage.

There is something about how the curve decays or dips, more specifically the rate at which it decays, which gives Exponential this unique property among continuous probability distributions. In fact, if you knew that time durations follow the Memoryless property you can work backwards and prove that the original probability curve has to be Exponential. 

As a contrast, other well known continuous distributions, say the Normal or Lognormal, do not have the Memoryless property. The Lognormal distribution is a more relevant comparison since, like the Exponential, it allows only values greater than 0, unlike the normal which allows negative values and is therefore not always appropriate for modeling time durations. In the Lognormal and Normal, the probability of a future event is not unconditioned by how much time has passed. This means that you have to keep track of the past when you calculate the possibility of a future event -- and this quickly gets very cumbersome and computationally expensive. 

For a further contrast, I've created a couple of roughly equivalent images, Figure 1 and Figure 2, for the continuous uniform distribution -- a relatively simple, bounded distribution with a flat curve. Here we see that the probability of an event happening in the next five minutes was originally 0.1667; after twenty minutes, it went up to 0.5.  

Among discrete distributions, the Geometric distribution has the Memoryless property.

3. Lifetime of a Device

I'd like to end the piece by raising a couple of questions. Probability textbooks routinely mention that the Exponential distribution can be used to model the lifetime of a device: time from when the device is put into operation to its failure. Here the Memoryless property seems puzzling to me.

If a device has worked for 3000 hours, the probability that it will work for another 1000 hours is exactly the same as when the device started operating. I find that quite amazing. Such a property is possible only if the failure of the device has nothing to do with wear and tear caused due to time. Otherwise, the longer the device works, the more likely it is to fail in the next time interval -- just as at the age of 70, the probability that we will die in the next 10 years is much higher than the same probability calculated at the age of 40. From what I've read, the lifetime of semiconductor components follows exponential time to failure distributions. But then how is it that these devices escape wear and tear caused due to time? And are there other examples?

Tuesday, April 14, 2015

The Traveling Moose # 6

I was reminded of the  traveling salesman problem (TSP) in an unlikely context, thanks to a display at the New York State Museum in Albany (click on image for better view). The display showed the zigzagging journeys made not by a salesman, but a radio collared moose, simply called #6. The moose seems to have traveled a lot, but the explanatory note on the display said otherwise: "Most New York moose have settled into limited travel routines...From February 1998 to November 2000 radio-collared moose #6 ranged over 350 square kilometers (217 square miles) east of Great Sacandaga Lake."   

Tuesday, February 17, 2015

A mobile surgical unit in Ecuador

Latest 3 Quarks Daily column is about my healthcare-themed trip to Cuenca, Ecuador last October. Full essay is here. This is how it begins: 
Since 1994, a small team of clinicians has been bringing elective surgeries to Ecuador's remotest towns or villages, places that have do not have hospitals in close proximity. From the city of Cuenca – Ecuador's third largest town, where they are based – the team drives a surgical truck to a distant village or town. Though a small country by area, the barrier of the Andes slices Ecuador into three distinct geographic regions: the Pacific coast in the west; the mountainous spine that runs through the middle; and the tremendously bio-diverse but also oil rich jungle expanse to the east, El Oriente, home to many indigenous tribes. Apart from a few major cities – Quito, Guayaquil, Cuenca – towns and villages tend to be small and remote.   
Isuzu Truck 2

Each year the team goes on 12 surgical missions, roughly one per month. A trip lasts around 4 days: a day's drive to get to the place; 2 days to conduct 20-30 surgeries (sometimes more sometimes less); and then a day to return. Patients pay a nominal/reduced fee if they can: the surgeries are done irrespective of the patient's ability to pay. The clinicians belong to a foundation called Cinterandes (Centro Interandino de Desarollo – Center for Inter-Andean Development).   
Amazingly, the very same Isuzu truck (see above) has been in use for more than 850 missions and has seen 7458 surgeries from 1994-2014! The truck itself is not very large; in fact, it cannot be, because it has to reach places that do not have good roads. The mobile surgery program has the lowest rates of infection in the country. Not a single patient has been lost. The cases to be operated on have to be carefully chosen. Because of the lack of major facilities nearby, only surgeries with a low risk of complication can be done. Hernias and removal of superficial tumors are the most common. Hernias can be debilitating, yet patients may simply choose to live with them for many years rather than visit a far-off urban hospital. For many, leaving work for a few days and traveling to get a health problem fixed is not an option.

A sky full of monarchs

A sky full of the famous monarch butterflies in Michoacan, Mexico. It's hard to capture this special phenomenon -- millions of butterflies congregating, after a 2000 mile journey from Canada and northern US, in a few fir/oyamel forests in Central Mexico -- on camera. This picture I took last week is not very good, but every black speck in the sky, however faint, is a butterfly. This year's monarch numbers seem to be better than last year, although still well below the average across two decades.

Monday, January 19, 2015

Birds seen this winter

It's hard to spot new birds during Massachusetts winters (I don't own a house with a yard or a bird feeder, which makes it doubly hard). The hundreds of species that make their home or pass through here are more easily observed in spring, summer and early fall. But last Tuesday – a bone chillingly cold but sunny day in Amherst – I ran into four species all at once. I had come out for a walk in a quiet part of town, a dead end street where an unpaved hiking trail leads to a pond. The unusually high levels of noise in the trees suggested that a lot of birds were active. The repeated deep thuds I was hearing indicated that woodpeckers were around, hammering on tree trunks.


So here are the species that I spotted, from left to right (picture assembled from Wikipedia images): the eastern blue bird; the black capped chickadee; the female downy woodpecker (the male has slight red marks on the head); the misleadingly named red-bellied woodpecker because the prominent red or orange patch is actually on the bird's curved head. The chickadee is the smallest of the four, and the red-bellied woodpecker the largest. Overall, nothing really surprising here – these are all common winter birds. But as an amateur bird watcher, I felt a special joy stumbling upon them; it felt, at least in those few moments, as if some special secret of nature had been unexpectedly revealed.

Some other things I've noticed this winter: (1) starlings, dozens of them somersaulting gracefully in the air in unison, literally a dance to avoid death, an attempt to disorient hawks that are hunting them (something similar to what's happening in this video. On a different note, the 150 million starlings in North America today are descended from the 60 odd European starlings that were deliberately introduced to New York's Central Park in 1890 by "a small group of people with a passion to introduce all of the animals mentioned in the works of William Shakespeare" -- talk about literature influencing ecology!); (2) young wild turkey, moving black specks from a distance, foraging in a snow covered meadow (here's a previous piece on wild turkey); and (3) a few weeks ago, at twilight, the mysterious, round faced barred owl, the only owl I've ever seen, well camouflaged against the bark of a tree, very similar to this picture.   

(This piece was cross-posted here.)

Tuesday, December 30, 2014

The resettlement of refugee farmers in East Punjab after Partition

A different version of this piece was published back in 2009, in the OR/MS Today magazine.

It is well known that the partition of the Indian subcontinent in 1947 had terrible consequences. Tens of thousands of people died. Millions were displaced and lost their cherished ancestral homes: Muslims left India for Pakistan, and Hindus and Sikhs left Pakistan for India. It was the greatest mass migration in history. But what is less understood is the manner in which the vast numbers of refugees were accommodated and settled into the newly divided regions. The greatest mass migration in history inevitably became the largest resettlement operation in the world.

How was this monumental task achieved? That question might take up many books, and perhaps many have already been written. But we get a glimpse of how it was done in the Indian side of the Punjab (East Punjab) in Refugees and the Republic a chapter in Ramachandra Guha’s post-independence historical narrative India after Gandhi. 

Guha’s chapter appealed to me for a different reason. My specialization is the field of operations research: the quantitative or optimization methods that are now used widely in the attempt to make service systems more efficient. The resettlement or land allocation problem set up by Guha in the chapter seemed to fall squarely in the realm of optimization. I was curious to know how the reallocation of land had been carried out in practice.  

Punjab was one of the partitioned provinces; the eastern part found itself in India while the western in Pakistan. A large number of Muslims had left East Punjab for Pakistan. But there was an even greater influx of Hindus and Sikhs into the east from Pakistan. Most of these refugees were farmers. Together they had abandoned 2.7 million hectares of land in Western Punjab but across the border in India where they now had to make a living only 1.9 million hectares had been left behind by Muslim farmers who had fled the opposite way. The problem was made more complex by three additional factors:
  • Each refugee family had a claim on how much they had owned prior to emigrating.
  • The fertility of the land differed; there were dry, unirrigated districts as well as lush, irrigated regions.
  • There were demands that families and neighbors be relocated in the same way as they had been in West Punjab. If possible entire village communities had to be recreated.
From the comfort of hindsight – and given how far computing power has grown in the last 6-7 decades – I can imagine formulating the land allocation problem as a large scale mathematical optimization model. The decisions (the x variables in the problem, which the model would be solved for) would be how much land to allocate to each family and where to allocate. The objective, expressed as  a function of the decisions, would be to minimize the difference between claims and actual allocations. The constraints would be equations that expressed limits such as the total land allocated could not exceed the total  land available. There could be other constraints to ensure that families and neighbors are relocated together. The mathematical model would also have to capture the spatial aspect, the variation in fertility and relate them somehow to the allocation decisions.

Even by today’s standards this is a very difficult optimization problem; it also has human or qualitative dimensions that are difficult to capture mathematically. The unenviable task of reallocating land fell upon the Indian government and its civil service workers. As a first step, they assigned each family of refugee farmers 4 hectares irrespective of its past holding; they also gave loans to buy seed and equipment. Viewed from an optimization lens, this 4-acre allotment is an “initial solution” – a feasible assignment to the decisions (the x’s) to get things going, but very far from optimal.

As families began to sustain themselves, applications were invited for them to claim more land, depending on what they had owned in West Punjab. Within a month, there were 500,000 claims. These claims were then “verified in open assemblies consisting of other migrants from the same village. As each claim was read out by a government official, the assembly approved, amended or rejected it.” Refugees tended to exaggerate of course, but were deterred by the open assembly method; if a claim turned out to be false they were punished by a reduction in land.

Sardar Tarlok Singh of the Indian Civil Service and a graduate of the London School of Economics led the rehabilitation operation. He used two simple but interesting rules (we call them heuristics) for allocating land, and this is where the pragmatism in the whole operation comes most clearly to light. Though claims had been filed, because of the reduced acreage, none of the refugees could be assigned as much land as they'd originally owned. Everybody’s claim had to be reduced by a certain percentage. Plus, there had to be some way of accounting for the differing fertility of land.

Sardar Tarlok Singh came up with two measures, the standard acre and the graded cut, which dealt with these issues: 

“A standard acre was defined as that amount of land which could yield ten to eleven maunds of rice. (A maund is about 40 kilograms.) In the dry, unirrigated districts of the east, four physical acres were equivalent to one standard acre; but in the lush “canal colonies” [where irrigation was strong], one physical acre was about equal to one standard acre. The innovative concept of the standard acre took care of the variations in soil and climate across the province.

The idea of the graded cut, meanwhile, helped overcome the large discrepancy between the land left behind by the refugees and the land now available to them – a gap that was close to million acres. For the first ten acres of any claim, a cut of 25% was implemented – thus one got only 7.5 acres instead of ten. For higher claims the cuts were steeper: 30% between ten and 30 acres, and on upward, so that those having more than 500 acres were taxed at the rate of 95%.”

With this rule, there clearly were losers, and the losers, of course, were those who had once owned huge tracts of land: “The biggest single loser was a woman named Vidyawati who had inherited land (and lost) her husband’s estate of 11,500 acres spread across thirty-five villages of the Gujranwala and Sialkot districts. In compensation she was allotted mere 835 acres in a single village of Karnal.”

It’s unclear what analysis motivated Tarlok Singh to come up with the specific ranges and the cuts. It’s possible that the taxing mechanism might have left too much land unassigned, and many claimants dissatisfied. Or, given that the overall reduction in total land was about 38 percent (the farmers had left 2.7 million hectares behind and now were being resettled on 1.7 million hectares), the taxing may not have been strict enough. The exact details are unknown.

What is known, though, is that by November 1949, a year and a half after the resettlement began, “Tarlok Singh had made 250000 allotments distributed equitably across the districts of East Punjab”. Even the soft constraints, such as settling families and neighbors together, were met to a large extent, though “the recreation of entire village communities proved impossible.” The resettlements were so successful that “by 1950, a depopulated countryside was alive once again.” Tarlok’s heuristics might have been simple, but they helped solve a complex, large-scale allotment problem in the aftermath of a traumatic event.

Halfway across the world, in the summer of 1947 — the same year that the partition of the Indian subcontinent took place — George Dantzig conceived the famous Simplex Algorithm. In the next decades the Simplex Algorithm, helped by advances in the computing capability, would provide fast solutions to large linear optimization problems. Each fall I teach the algorithm to graduate students from many disciplines. But in 1947, the Simplex algorithm could only tackle a small version of the classical diet problem: optimally choosing food items for a family to minimize costs while ensuring minimum nutritional needs are met. That particular diet problem had only 9 constraints and 27 variables, but took 120 man-days to solve; worksheets were glued together and spread out like a tablecloth to assist in determining the optimum solution [1].

This puts the enormity of resettlement problem in perspective. With half a million people making claims – which means there would be hundreds of thousands of variables and constraints – even an awareness of Dantzig’s algorithm could not have helped the Indian Civil Service.

Nearly 7,000 officials were needed for the resettlement effort; they constituted a refugee city of their own. The problem occupied them for a period of three years. Imagine the paperwork and the records that had to be kept and retrieved; imagine the disputes among the refugees, the flared tempers and the jealousies. But imagine also the perseverance of everyone involved. It made me think about what it is exactly that makes certain large scale operations successful and others not. It’s hard to make generalizations, but one feature seems to be effective coordination across large groups of people: largely error-free lunch deliveries by the Dabbawalas of Mumbai is an example that comes to mind.

Related notes:

1. Guha ends the chapter in his book poignantly. The resettlement, Guha says, may have been successful, but the general sense of loss could not be undone. The migrating Sikhs had left behind a beloved place of worship, Nankana Sahib, the birthplace of the founder of their faith, Guru Nanak. Muslims migrating from East Punjab too had left behind the town of Qadian, the center of the Ahmadiya sect of Islam; the Ahmadiya mosque was visible for miles around. Very few Muslims now lived in Qadian, which was full of Hindu and Sikh refugees. Guha quotes the editor of the Calcutta newspaper Statesman, who wrote that in both Qadian and Nankana Sahib there was “the conspicuous dearth of daily worshippers, the aching emptiness, the sense of waiting, of hope and…of faith fortified by humbling affliction.”

2. The picture shows a boy at a Delhi refugee camp in 1947. Here is the source. The largest refugee camp, though, was at Kurukshetra, consisting of nearly 300,000 people. For their entertainment, film projectors were brought in and Disney specials featuring Mickey Mouse and Donald Duck were screened at night. It was, as one social worker described it, a “two-hour break from reality”. 

1. Bazaraa, M., Jarvis, J., and Sherali, H., 2005, Linear Programming and Network Flows, Wiley 3rd Edition

2. All quoted parts in the piece are from Ramachandra Guha’s India After Gandhi

Tuesday, December 23, 2014

The Undocumented Journey North, Through Mexico

Latest column is up. A bit long, some typos here and there, but hopefully still interesting. Here is an excerpt: 
From 2000-2006, I was a graduate student at Arizona State University in the Phoenix metro area. My neighborhood, a ten minute walk from the university, had cheap apartments where Asian students lived alongside immigrants from south of the US-Mexico border. We students had visas, had made safe journeys on flights, and now worked and studied on campus. Many Hispanic immigrants, in contrast, had made life threatening journeys and had crossed the border illegally. They now did construction, farm, and restaurant jobs for a living. At the neighborhood Pakistani-Indian restaurant, I remember seeing – through a decorative window shaped as a Mughal motif – three Hispanic workers in the kitchen patiently chopping the onions and tomatoes that would go into the curries that I enjoyed.
Some Indian students looked down on these immigrants, blaming them for petty bicycle thefts and how unsafe the streets were at night. And just as all East Asians were "Chinkus", the immigrants from south of the border were "Makkus" – a twist on "Mexican", used mostly (but not always) in a negative sense. No one, though, had a clear sense what the stories of these immigrants were. While it is true that a large percentage of those who cross the border are from Mexico, tens of thousands each year come from the troubled countries further south – Guatemala, El Salvador, Honduras. This year, an estimated 60,000 unaccompanied minors from Central American countries, fleeing violence in their home towns, will cross the border. Surprisingly, even hundreds of undocumented South Asians cross via Mexico – but more on that later.
More here. Towards the end, I talk of the South Asian angle, including a Bangladeshi man I met in Quito, Ecuador, hoping to make the undocumented journey to the US: 
The South Asian Angle: A 2009 report by Homeland Security estimated that India was number six – after Mexico, the Central American countries, and Philippines – with 200,000 undocumented immigrants currently in the US. Between 2009 and 2011, 2600 Indians were detained by the Border Patrol along the US-Mexico border. This coincided with a visa-on-arrival policy that Guatemala and other Central American countries allowed for Indian passport holders. So Indians could fly in to Guatemala City and start the journey north. The cross continental flight suggests Indian migrants had more money than most Central Americans - perhaps the money bought them a safer passage. Guatemala has now stopped visa on arrival for Indians. 
A Bangladeshi in Quito: In October this year – in one of those strange, unlikely encounters that happen during travel – I met met a Bangladeshi man selling samosas, 3 for $1, in the old town of Quito, Ecuador. He was the only South Asian among many Ecuadorian street vendors. The samosas were in a container - perhaps a hundred of them. Business was brisk. To appeal to the locals, he had cleverly called the samosas "Empanadas de India".  
I spoke with him in Hindi. He had been in Ecuador for five years and was fluent in Spanish. Life had been reasonable, he said; accommodation, food and cost of living were inexpensive in Quito. He was in a position now to apply for an Ecuadorian passport. But his interest had always been in migrating to the United States – the same route via Mexico that others take. Some acquaintances of his had made it there already. He asked me about jobs in the US. Unfortunately, I had no concrete sense on what an undocumented immigrant could expect. I did tell him that the journey through Mexico, from what I'd heard, was risky. But he seemed intent and had a more optimistic view. All this was before I read Martinez's book -- if I had known of The Beast, I would have urgently recommended it to him. 
How unusual and compelling his story is: here was a man from Bangladesh in, of all places, Ecuador, biding his time patiently, saving up money by selling that most South Asian of snacks, samosas, and even securing a backup citizenship, so that he could risk the journey north! 

Monday, December 01, 2014

A 1000-year old witness

History in the rings of a 1021 year old redwood tree -- from Muir Woods National Monument, north of San Francisco. The tree was born 930 AD and fell in 1930, and was alive during key events in North American history: a millennium of great tumult, especially since 1492. Each year produces a natural growth ring, varying in size depending on the type of year (dry, rainy etc.) so there are approximately 1021 rings in this cross section. Redwood trees are massive (350 feet tall), which is not conveyed by these pictures. Sorry for the poor images and the flash -- the forest created by these tall trees is dark even during the day, and my cell phone has poor resolution.

Thursday, November 20, 2014

Essays at 3 Quarks Daily

If you've been wondering why I haven't posted here for so long -- well, it's not because I've been busy with work. In fact it's been a good year for writing essays. I've been writing this year for a website called 3 Quarks Daily since this January. So far I have nine essays, all collected here. Comments welcome! I should be cross posting the essays here too, but I've been lazy. Maybe, with the 3QD pieces coming regularly, I'll find some way to post more informally here. Though I've probably lost what little readership I had. 

Tuesday, October 22, 2013


On July 12th this year, after three days in Istanbul, I traveled with my friend Serhat, to Erzurum, a regional city about 770 miles away in Northeastern Anatolia.

The flight was 2 hours long. After takeoff, the plane first took a northwards course: the dark blue waters of the Black Sea were beneath us. A few minutes before, I’d glimpsed the 32-km-long Bosphorous Strait, Istanbul’s iconic landmark. Istabulites know the maritime significance of their city very well, but to me it was a revelation: a ship traveling from Odessa, a Ukrainian city on the north cost of the Black Sea can travel via the narrow Bosphorus to the Sea of Marmara; and from there through the Dardanelles Strait to the Aegean and Mediterranean Seas; and finally through the Strait of Gibraltar to the Atlantic Ocean.  

The marked spot to the right of the map indicates Erzurum

The plane eventually steered eastward, following Turkey’s Black Sea coast, and during the last half hour, it turned inland to northeast Turkey. The landscape was consistently mountainous: sometimes lush green (especially when close to the Black Sea), sometimes covered with cloud, sometimes dry, the ridges on the slopes of brown mountains casting shadows in the late afternoon light, creating a distinctive visual texture. 

Erzurum, a town of about 367,000, lay in a sprawling plain at the base of one such dry mountain range (Mt Palandöken is a ski resort near Erzurum). A haphazard checkerboard of farms stretched for miles and miles around the city. Many of them, I discovered later, were hay farms, important in a region whose economy depends heavily on stock breeding  We rented a car at the airport. The small airport, the plains around and the mountains in the distance reminded me a little of Bozeman, Montana. On our way to Erzurum center, we passed by the gates of Ataturk University. With 30-40,000 students and medical school to boot, this is a major university and contributor to the economy. 

By the time we had checked into the Esadaş Hotel, along Cumhuriyet Caddesi, Erzurum’s main thoroughfare, it close to iftar time; light was fading fast and the Ramazan fast would soon be broken -- at 7:53 pm.  We started walking to the popular Gelgör Restaurant.  On the way, we passed by two historic mosques: the Yakutiye and Lala Pasha. Erzurum, like the rest of Anatolia, has seen many layers of history: it has been influenced by Greek, Roman, Arab, Persian, regional Georgian and Armenian Christian, Seljuk and Mongol rulers.

In the courtyard of the Lala Pasha, we ran into two boys, aged between six and ten. The younger one was selling toilet paper or rolls of tissue neatly folded in a plastic cover; the older was carrying some small contraptions, one of which looked like a low plastic bench.

We did not buy anything, but Serhat got to talking with them. He told them that I was from Hindistan. Almost immediately, the boys started repeating a few words frantically to me. The younger one said, “Amita..bhaccha” at least five times, before I realized they were referring to Amitabh Bachchan. The older one was saying Shahrukh Khan in his own way. Bollywood’s popularity in unexpected places is not unusual --  from West African taxi drivers in Minneapolis, to painters on the streets in Lima (Peru), to an Uzbek man I met on a Grand Canyon hiking trip: everyone is familiar with Bollywood. The bigger surprise was that these kids, making do with basic Turkish, were not locals but from Kabul, Afghanistan. Serhat learned that they had entered Turkey illegally in what must have been a very long journey from home.

Just then there was a loud explosion and puff of smoke: this was the city cannon signaling the end of the fast. Prayers immediately reverberated from the minarets all around. When I looked at the twilight sky above, I saw large numbers of swallows emitting low shrill sounds and flying very fast like quivers of arrows – their excitement probably had nothing to with the excitement of a Ramazan evening, but in my mind at least it seemed so. The fact that I was traveling in a predominantly Muslim city in a far corner of Anatolia had until then only been a fact. But the impressions of that evening – the unlikely meeting with the kids from Kabul; the firing of the cannon; the azans; the swallows – all came together with special force to make that moment personal.  


We continued to walk towards the restaurant. Ramazan is a time to be with family and friends, so the streets were completely deserted, and this reminded me of the bleakness of American suburban neighborhoods. But Gelgör was bustling with people relishing their kebabs delivered non-stop on skewers by busy waiters. Here it was easy to feel the festive, communal atmosphere of Ramazan.

After dinner and a rich dessert – the kadayıf dolması – we walked through the streets and alleyways of Erzurum, and came across more old tombs and mosques. The emptying out of streets at iftar time had given the impression that the night life of the town was over. But after 9 pm the streets got busier and busier.

Tea houses are a distinctive feature of Turkish life: male Turkish life, if you are in a conservative town. In alleyways and the main streets of Erzurum, I saw plenty of informal, open-air tea houses: large, stylish and what looked like stainless steel samovars (heated, in one case, atop a hearth with wooden sticks); men chatting with other men, tea cup in one hand, cigarette in the other; dark red tea in glass cups pleasing to the eye. Even the little cubes of sugar provided on the side have aesthetic value. This tea habit – one cup is never enough and each cup costs less than a lira -- reminded me Dostoevsky and Tolstoy’s novels, where in the large gatherings, parties or soirees of the aristocracy, the presence of tea served in samovars is mentioned without fail, often at the expense of other items (Dostoevsky hardly ever describes food items other than tea, which left me to wonder if Russian food at the time was very dull).

Back at Cumhuriyet Caddesi, which runs through the city center, families – plenty of women and children – were out in full force; the noise and the traffic was incredible, given how late it was. Near the Yakutiye and Lala Pasha mosques, a stage had been set up and there would perhaps be skits and other entertainment. Glass-fronted dessert and ice-cream shops were doing quick business. There were billboards advertising stylish and expensive Islamic wear for women: the elegant black dresses and ornamented head wear had a touch of modern fashion in them even if their basic function was conservative. Overall, Erzurum conveyed a sense of prosperity and wealth.    


There were many questions I had about Turkey and Erzurum, and Serhat did his best to fill me in. He pointed out the billboard of a radical Islamic party with the motto, “Morality and Spirituality First”; the party wanted to appeal to all Turkic peoples of Central Asia. This affinity to the broader ethnic group stemmed from history: the Turks as a people were originally from someplace in south Siberia; they had slowly, over a millennia or more, made their way westwards, interacting with many other cultures along the way -- borrowing loan words from the Persians and the Arabs (this is perhaps why many Hindi and Turkish words mean the same thing, because India too was ruled by Central Asians). The westward movement of the Turks finally culminated in the creation of the Ottoman Empire. In Eastern Anatolia the Seljuk Turks, whose architectural remnants are a major attraction in Erzurum, were prominent a few centuries before the Ottomans arrived on the scene. 

This idea of Turkic groups conquering new lands raised some issues. How did the rulers bring the original inhabitants of Anatolia, who would have had their own diverse traditions and languages, into their fold? I was also curious how, in the early 20th century, the Turkish state had been fashioned by Ataturk, especially this far away in Anatolia, and the tensions inherent in the transition from an empire to a nation-state. I had questions, too, about the predominance of a single language and religion in Turkey. To my eyes – perhaps because I had grown up in India – Turkey seemed remarkably homogeneous, but I knew that couldn't be entirely true. Whatever its history and politics, Turkey seemed to have done much better than India in some essential aspects: its regional cities were cleaner, more organized and better equipped in terms of infrastructure. 

To the questions on history, I found some partial answers in the Rebel Land, a non-fiction book -- a very personal one -- by the English correspondent Christopher de Bellaigue. About a 5-10 years ago, Bellaigue visited the seemingly nondescript town of Varto, 3 hours south of Erzurum, for extended periods, in the attempt to unearth “the riddle of history in a Turkish town”.

Fluent in Turkish, Bellaigue was able to talk to the town mayor, civil servants, army men, businessmen and shepherds. In the process he unveils a complex and tangled history bringing to fore fault lines in modern Turkish history: the Kurdish question; the Alevis who were at odds with the majority Sunni Muslims; the Armenian mass deportation and killings that had happened in the chaos of shifting alliances in the First World War, a time when the Ottoman Empire was on its last legs, its Christian subjects in Europe had become nation states, and Russia was advancing into Ottoman territory in Eastern Anatolia.  

In an early chapter of Rebel Land de Bellaigue is still deciding which town he should choose for the book.  In Ankara, he meets a Kurdish friend for dinner. The friend tells him about Varto, “a small place in the southeast… but not far south as to be caught up in regular fighting…a little north of the great Armenian monastery of Surp Karapet.”  De Bellaigue’s friend had “got to know Varto through his wife, also an Alevi, who had relations there.  The Alevis of Varto generally spoke Zaza, he said, and the Sunnis Kurmanji; both were Kurdish languages”. He then said that the “Alevis of Varto suffer from a peculiar existential angst. They are divided over whether they are Turks or Kurds.” 

These were precisely the sort of nuances I had no idea about. Recently, a native of Erzurum, now living in the US, confirmed how her home town stood out sharply as a Turkish Sunni bastion even among the generally conservative towns of Eastern Anatolia. In a few days, I would visit the much smaller and poorer town of Kars, close to the eastern border with Armenia. Almost immediately after getting off the bus, I could tell that Kars was messier but more diverse and relaxed in its outlook. Maybe I was biased; maybe I felt that way because there were many good places, with vegetarian options, open for lunch in Kars; lunch during Ramazan at a restaurant in Erzurum does not seem to be possible.

After that evening in Erzurum, Serhat and I left early the next day. We drove north through the mountains, to the town of Rize – the hometown of the current prime minister, Erdogan – on the Black Sea Coast. From there, we headed east towards a group of villages (part of a United Nations Biosphere) located in a lush green, mountainous region on the border between Turkey and Georgia. After twelve hours of driving we finally got to our accommodation at 8 pm. I will describe this journey by road in my next post.  

Tuesday, September 03, 2013

Some analyses of 40-year bird count data in Central Massachusetts


Two years ago, in a post with a cheesy title, I’d written about seeing cardinals frequently. I continue to come across them: it is still a thrill to see a sharp movement of red in the branches of trees, and to hear the bird’s distinctive calls. Inspired by these sightings, I find myself now drawn to other birds, mammals and insects that live in the wooded areas of New England.

On May 27th this year, I visited a tract of land preserved by the Massachusetts Audubon Society near Worcester: the Wachusetts Meadow Wildlife Sanctuary. There, I met naturalist Joe Choiniere. We talked for a while. Joe knew well the birds in the area -- details of their range, habitat, prevalence, appearance, calls etc. As the current property manager, he, like others before him, organizes a bird count each year at the meadow. I told him about my interest in looking at the data -- an interest that has grown ever since I began to teach probability and statistics. 

Joe said could share the data with me.  To observe how the data was collected, he also invited me to attend the count this year, on June 9th. It was a somewhat cloudy Sunday morning, but visibility was good. Around 10-15 people, most of them expert birders, broke into groups and went on different trails that wound through the hill forests, ponds and marshes of the sanctuary. I followed one group along the main dirt road. Each person carried a paper listing of all species and marked the numbers as soon as something was spotted. Close to 500 species of birds have been observed in Massachusetts, so one has to be a keen and experienced observer of birds to get the identifications right.

Late morning, everyone returned to a room in the Visitor’s Center. A coordinator called out the name of each species; the birders around the table responded with their counts, and a tally for 2013 obtained. This process remains the same each year. Minutes of the discussion are maintained, and journal published each year presents the tally. Variables such as the number of observers, weather and temperature can of course change somewhat, but the trails on which the counts are made have mostly remained the same. Most importantly, despite year to year variations, the 50-year duration means that some general inferences on population levels of particular species can be made. 

Joe sent me an Excel Spreadsheet with the count for all species at the meadow from 1964 to 2003 (the data for 2003-2013 is also available, but has not yet been entered in Excel). I started working with the data recently and made some preliminary graphs. In this post, I will illustrate, with examples, whether or not the annual bird count at Wachusetts Meadow tallies with other statewide trends. 

So let’s take a look at the cardinal count for 40 years since 1964, when the count first started at the meadow. The x-axis is the year. The y-axis is the number of cardinals observed by birders at Wachusetts Meadow.  We notice the variability from year to year (except in the early years when no cardinals were seen). But it’s pretty clear that there is an increase in the number of cardinals seen, even though there are still years when none are observed.

Note that the image is only of the male cardinal (somewhat sexist, you could say!); the female is more gray than red, but females are of course part of the count.
What do the statewide counts compiled by the Massachusetts Audubon tell us on the prevalence of cardinals? There are three such counts: the Bird Breeding Atlas (conducted from 1974-1979 and again from 2007-2012); the Bird Breeding Survey (an annual effort on specific roadways); and the Christmas Bird Count (a 114 year tradition that is kept going by enthusiastic citizens).  Trends in the cardinal population based on these counts can be seen here. Notice that all counts suggest either a "likely" or "strong" increase in the number of cardinals. This agrees with the graph above, based on the Wachusetts Meadow annual count.


It is of course also possible that Wachusetts cardinal count may have only coincidentally agreed with other counts. So, for further validation, let’s look at the 40-year trend at Wachusetts for two other birds: the cliff swallow, and the house finch. 

I am fascinated by swallows: this July, I saw hundreds of rock swallows, their appearance similar to the above image, at the desolate ruins of the ancient city of Ani, along the river that separates Turkey from Armenia.
Sadly, for the cliff swallow we see a precipitous decline and counts have been zero for most of the 80s and 90s. This agrees with three statewide counts posted at the Massachusetts Audubon Society: they confirm a “strong decline” in numbers. The brief note says: “Today, the Cliff Swallow occupies less than half of the distribution it held in 1979. Loss of nesting structures, such as old barns and bridges, along with nesting competition from introduced House Sparrows are among the factors accounting for this restricted distribution.” 

For the house finch, we see no sightings at all until the 1980s. It turns out that the house finch was introduced into this region in the previous decade: “The introduced House Finch arrived in Massachusetts during the 1970s and never looked back. It can now be found living alongside humans over much of the state, and as a breeder it is nearly ubiquitous.” In the above figure, we don't notice high numbers in the years leading up to 2003, but the early years validate well with the available knowledge on house finches.

In both these cases, the Wachusetts Meadow trends again seem to match reasonably well with other statewide trends. Still, this is just preliminary evidence. Birders may be influenced by what they hear from others in their community or what they read in journals; during counts, they might unconsciously seek for a particular species or ignore others, thus introducing some “unnatural” variation. Yet, there is no way to completely escape such biases in a field study.

What impressed me most was the perseverance of anonymous individuals participating to keep such censuses alive all over the country. The comprehensive bird counts listed in the Massachusetts Audubon Society, all depend on the efforts of such individuals; the Christmas Bird Count has been going on for 114 years! 

Wednesday, August 14, 2013

A trip to Turkey


In July this year, I traveled to Turkey. I reached Istanbul on July 09 – the day Ramazan began – and stayed in at Koc University near Sariyer, a northern suburb of the city on the European side (pardon the incorrect Turkish spelling: for those used to the English alphabet, the Turkish one is convenient and familiar, but at the same time there are some key sounds the English alphabet does not cover). I was attending an academic healthcare conference at Koc. But that was merely an excuse to begin again the kind of unstructured travel I’d managed in Latin America a few years ago.

I stayed for 3 days in Istanbul but then flew with a longtime Turkish friend, Serhat, to northeast Turkey: cities, towns, and villages in Erzurum, Artvin and Kars provinces, along the border with Georgia and Armenia. Serhat, an avid traveler himself, had already been to these and other far flung parts of the country. So this trip was different in that I had somebody who could translate and interpret things for me. I returned to Istanbul on July 18th, and experienced the Ramazan rhythms around the Sultan Ahmet mosque that evening. The next day I left for Boston. 

Why Erzurum and Kars? The choice was arbitrary; I wanted to get a sense of a completely different part of Turkey. And the Northeast seemed like a gateway to the vast Eurasian mountains and steppes of the Silk Route that I knew very little about, but had always had dreamed of traveling to. This post provides some quick impressions of my time in Istanbul, but in the coming weeks, I’ll write about the 6-7 days of extensive travel in Northeast Turkey.  


On July 9th, my flight arrived at Istanbul at 2 pm. It took about 40 minutes to clear my passport; the line for South Asians was the same as that for Iraqis, who outnumbered all other nationalities. The Iraqi lady behind me wondered if I too was from Baghdad.

Serhat picked me up from the airport. We used the metro, tram and funicular – I was impressed how organized and cheap public transportation was – to get to Taksim Square and Gezi Park, the site of fierce public protests and even fiercer police/government retaliation this summer. Gezi Park had just been reopened to the public the day before. But the reopening had prompted more protests; teargas had again been used the previous evening.

One view from Taksim: the image of Ataturk, the founder the modern Turkish state, on the building in the distance. 

Police vehicles at Taksim
Taksim was quiet when we arrived: it felt strange that the explosions, fires and water cannons that I’d seen on television in June had happened here. But for the police trucks that were parked in one corner, ready to repress if a critical mass of people gathered, Taksim seemed as safe as the rest of Istanbul. I passed through Taksim twice again in the next days with no problems. I was told that the story was different the evenings, when those interested in protesting got off their day jobs began to congregate at Gezi Park. 


I did the usual things that tourists do in Istanbul: visiting buildings, mosques, streets and bridges the city is famous for; taking the ferry ride on the Bosphorus Strait (we took a public ferry used by commuters in the evening: the 80-minute ferry ride from Eminonu to Sariyer, as scenic as any I’ve been on, astonishingly cost a mere 1.5 US dollars); visiting Orhan Pamuk’s Museum of Innocence, an actual museum based on the novel of the same name, in Cukurcuma, near Istiklal Street; and shopping for food, spices and souvenirs in and around the Grand Bazaar. There’s plenty of information about these things in travel books, so I won’t say anything more, except that Istanbul lived up to its promise.

Vegetarian options at a restaurant on Istiklal Street. 

Courtyard of the Sultan Ahmet Mosque at 10:30 pm

When I first reached the gates of Koc University, I had to check in with the security. This private university – owned by one of Turkey’s prominent business families – has a secluded campus set on a hill in the midst of pine and oak forests, far north of Istanbul’s attractions, and close to where the Bosphorus opens into the expanse of the Black Sea.

A security guard in fatigues asked for my passport. He said I’d have to wait for the next shuttle that would drop me near my dorm accommodation inside campus. Until then I could spend time inside the small security room. It was around 8:45 or 9 pm. About five security guards were absorbed in having dinner in the room. A number of dishes were on display in plastic or glass containers: bread, salad, fried cheese rolls, pasta or noodles, vegetables, rice. All dishes were shared. The guards offered me some of the food. I ate a cigar-shaped fried roll filled with slightly salty cheese. The guards did not see it, but I am sure my eyes must have lit up at the taste. 

I remember feeling struck by the atmosphere in the room: there was something very warm and genuine about that gathering. It didn't occur to me then, but this was of course the dinner, iftar, after the day-long Ramazan fast: perhaps even more special, since this was the first day of Ramazan, so the first breaking of the fast. Later, in Northeast Turkey, while sharing tables with strangers in crowded restaurants – all customers patiently waiting for the powerful microphones on minarets to signal the end of the fast, even as freshly served bread, soup and salad sat enticingly on the table – at these restaurants, I experienced again and again, that same atmosphere of a communal meal.