Brewing storm on cloud gaming. Are CDNs the saviors?

Cloud gaming has the potential to become a revolution in the way games are developed and distributed. Instead of requiring end-users to buy powerful computers to play modern games, cloud gaming performs the game computation remotely with the resulting output streamed as a video back to the end-users. Cloud gaming is thus expected to meet the demand of both gamers (who want platform independence), and game developers (who want to reduce their development cost and to gain flexibility in game updating). The enthusiasm has however been severely chilled when the main actor in the area, namely OnLive, ran out of money.

This debacle is not surprising. Many of cloud computing’s core design tenets conflict with cloud gaming. Cloud providers only offer general purpose computing resources that are located in a relatively small number of large data-centers. Unfortunately, these architectural decisions are in conflict with the needs of cloud gaming, which are interactive (hence highly latency-sensitive), and require specialized hardware resources, such as GPU and fast memory. Furthermore, many cloud data-center locations are chosen to minimize cooling and electricity costs, rather than to minimize latency to end-users.

Despite these drawbacks, many analysts still believe in cloud gaming. In a near future, the number of users served by physical machines should grow, data-centers should include GPUs, game engine should be re-designed… OnLive may just have been too early.

Still, the question of the number and the location of data-centers remains. Past studies have found that players begin to notice a delay of 100 ms. However, at least 60 ms of this latency should be attributed to playout and processing delay. Therefore, 40 ms is the threshold network latency that begins to appreciably affect user experience. 

In a recent academic paper, which was presented during the ACM Netgames conference, we have performed a large-scale measurement study to determine :

• the percentage of population that can be served with today's cloud infrastructure. With EC2 infrastructure, less than 40% of population can play highly interactive games (network response time 40 ms), and only two third of population can play the least demanding games (network response time 80 ms). See Figure below.

Population covered by EC2 cloud infrastructure

• the number of data-centers that are required to have a decent population coverage. Unfortunately even if 20 data-centers are deployed, less than half of population would have a network response time of 40 ms.
• the gain of using a CDN infrastructure. They are significant. Today's CDN servers do not host GPU, and they are not designed to serve a very small number of users (only 8 users per server with state-of-the-art technologies). But who knows what will be CDN next strategical moves? Our study shows that embracing cloud gaming is a very good idea for CDNs, isn't it?

Population covered if EC2 is augmented with CDN smart edges

Lessons learned at UWaterloo (2nd part): research organization

Here is the second post about my experience at University of Waterloo. After the ode to the co-operation education program, here is another positive observation related to research organization. All in one, I have the feeling that the time spent in meetings by researchers in North-America is four times less than their European counterparts. I wish statistics could support this claim. Why so? I identified at least two explanations.

Firstly the structures fundamentally differ between American and European research institutions. It might sound like a caricature, but Americans promote individual successes while Europeans build large corporations. A research department in an American university is an aggregation of independent researchers, who manage their own team of students and research fellows with their own budget, and who develop their own line of research. In Europe, senior researchers should gather into so-called research teams, which are expectedly consistent. European teams should define a strategy, share budgets and generate activity reports to justify they still deserve to exist. They are regularly challenged by numerous "administrative research managers", who are no longer researchers, but whose job is to "organize". Obviously, I think that the model where a department is like an incubator of entrepreneurial researchers is the right model. American researchers focus on their team, spend time on their own activities and are committed to succeed in academy by any mean. European researchers waste their time in meetings and in bureaucratic activities. Furthermore, the former model enables paradoxically better collaborations among researchers because these spontaneous collaborations are not based on any explicit agreement.

Secondly, research funding target individuals, not collaboration. Europe is crazy about "calls for collaborative projects". Our beloved European funders seem convinced that the only way to do research is to make people work together on a well-defined topic. European companies interested in academic research contribute to research through collaboration in these projects. I will again exaggerate a bit, but collaborative projects are not the norm in America. Researchers get small amounts of money from companies through direct grants, which favor transient, short, focused cooperation. And they also receive individual grants from their public funding agencies. Such model does not force researchers to waste a significant amount of time at collective writing, synchronization meetings and expense justification. Ask American researchers who experienced European projects if they want to do it again. Their answers are likely to be harsh about this crazy administrative nightmare.

I found this research organization far more efficient since it allows researcher to focus on their core activities. And I am afraid that the situation gets especially worse in France because I see a growing number of "administrative managers" who gravitate around the academic world. They are expected to be "interface" between researchers and funders, but their job (to organize researchers and to set up "calls for projects") actually interfere with researchers.

Lessons learned at UWaterloo (1st part): the Co-operative Education program

I spent one year at University of Waterloo in the Electrical and Computer Engineering department. I will try to extract from this fruitful experience a small set of short lessons. Here is the first one, an ode to the so-called Co-operative Education system.

What Waterloo calls a co-operative education system is a dual education system, which combines academic studies and professional works. We also have such program at Telecom Bretagne (my employer), see this link (only in french). All undergraduate programs are "co-op" at Waterloo. According to rankings based on surveys, it works pretty well for them (#29 for Business Insiders).

In France, dual education system at the graduating level is not the most prestigious curriculum. The "royal way" consists of two harsh years focusing on abstract mathematical concepts, a success in a ultra-competitive exam, then three years in an engineering schools at socializing (i.e. partying) and specializing. Students enrolled in dual education system are not considered as the best because they have not demonstrated outstanding scholar skills at the ultra-competitive exam. I already expressed serious concerns about such curriculum in a modern (i.e. computer oriented) society.

On my side, if I had to hire one engineer in my research team or in a start-up, I would definitely hire a "dual education" engineer. As far as I can see at Telecom Bretagne and UWaterloo, students have a lighter scientific background on the fundamental areas. However, they just program well. And they just work well. I have to admit that, like probably most companies, I value "programming well and working well" far higher than "having a strong background on fundamental scientific areas".

Dual education is an efficient way to teach software programming and project management, since, in general:

• Teachers don't code. You can't count on them to learn programming tricks.
• Teachers can't catch up all new technologies. Nobody is an expert in MapReduce, Ajax, and ObjectiveC at the same time. You can't expect that from teachers neither.

During academic terms, teachers can focus on the fundamental of programming and they can skip the courses about languages and technologies. During their terms spent in companies, students can discover the latest technologies and code with professionals. Good match, ins't it?

On the pivotal role of post-doc in research groups

A sabbatical is a great opportunity to study the internal process of other research groups. For a young European scientist like me, there is much to get from observing what American professors implement for the management of their large teams. In general, academic people are reluctant to apply industry-like management processes, which are supposed to prevent groups to be creative and lively. However, the research is becoming objective-driven, and the raise of grant agencies and project-oriented funding makes that an unexperienced young professor can now receive enough money to hire a large team of researchers. There is a need to address the tabooed topic of research group management.

A unique characteristic of research groups is the huge gap between a graduate "master" student, who has some scientific background but actually knows nothing, and a senior post-doc, who is supposed to be a peer, fellow researcher. One of the purposes of a research group is to assist the development of new skills… and to let people leave when they are, at last, autonomous and efficient.

It is my understanding that post-docs are the most challenging to work with. Some of them are super-PhD, while some others are rather mini-professor. Most of them do no longer require to be "tutored", but they still have to acquire some critical skills. In particular post-docs should ideally develop their own ideas from scratch, tutor younger students, actively contribute to industrial collaborative project, and expand their professional network.

On the one hand, delegating is dangerous for a professor. If a collaborative project fails, the one that will be directly affected in the future is the professor, not the post-doc. If young students spend too much time working on barely publishable ideas, utilize old-school technologies and miss latest exciting papers, it is the professor who eventually has to make the student catch up, not the post-doc. On the other hand, delegating is a real chance to expand the research group, to work on new areas, and to offer the opportunity to the new post-doc to acquire critical skills/experiences she misses.

For these reasons, the management of a post-doc is very touchy. On my side, I consider the following:

• before opening a post-doc position, I will clearly define my needs. I identified at least three critical needs that might pop up someday, and that would justify opening a post-doc position:
• I want to be assisted for tutoring young students
• I want to partly delegate the management of a heavy collaborative project
• I want to explore a well-identified brand new topic
• once my needs identified, I hope that finding a matching post-doc will be easier. Compared to my previous experience, I will pay more attention to the actual motivations of candidates, and I will pay less attention to their background or their past achievements. Of course it is important to know whether the candidate can write paper or produce experimental stuff at the expected level of quality for a top academic conferences, but it is even more important to know what is the status of the candidates regarding the need.
• then, if objectives are clear and well-understood on both sides, I guess the collaboration shall be more productive.

About faculty positions in Europe (and especially in French Grandes Ecoles)

I hear a lot of complaints from post-docs and near-end PhD students about the quasi-impossibility to obtain a faculty position in North America today: too few open positions and too many very strong candidates with impressive publication history and dithyrambic recommendation letters. When I ask these frustrated candidates "Do you consider positions in Europe?", they usually look at me strangely. Europe?

Since we have an open tenured faculty position in my lab, I think it is time to explain what a faculty position means in Europe / in France / at Telecom Bretagne.

Life for faculties in Europe. The vast majority of universities (and research centers) are located in the downtown of major cities. There is no university in the middle of nowhere in Europe. A typical city hosting an university is a major city in the sense that such city concentrates everything you need for a reasonably good urban life: shops in downtown, cultural events, and good public transport (metro, train, airport). In cities inhabited by more than 300.000 people, you can find everything, especially services and jobs. Smaller cities are generally also lively nice places to live. For example, the city of Brest (France, 170.000 people, 300.000 with suburbs) is incomparably more lively than Waterloo (Ontario, aggregated population of 400.000 people with Kitchener and Cambridge).

There is of course a language issue, since countries like France, Italy or Spain are still not exactly english-friendly. I think it is no longer a major issue for teaching. Most good universities have developed english-based courses, and recruiters prefer now a strong international candidate to fluent ones. Besides, I have three comments: (i) things change, the ratio of people speaking english increases, Europeans themselves travel more and get used to dealing with people from abroad countries. (ii) from my experience, kids are fluent in a new language in less than one year. It might be harder for parents, but people with a decent social life should be able to converse in a couple of years. (iii) multilingualism is one of the key assets of Europe. It is a chance for curious intelligent open-minded people. And faculties are expected to be curious intelligent and open-minded.

One of the major drawbacks of Europe is that every country has its own recruitment process. Since 2007, all European universities have the same curriculum. I think it is just a question of time that higher Education policies will be harmonized as well. But it is currently the main issue, typified by the crazy French system.

Faculty positions in France. All faculty positions in France are tenured: once you get hired, you can stay here forever. Moreover, the life in France is among the very top worldwide (in particular weather, cultural life, infrastructure and service quality). The combination of the above should place France as the first location for young scientists from all over the world. France however suffers from two critical drawbacks.

The first drawback is the recruitment process, which has been created by French for French. It is a ridiculous Kafkaian system, which actually prevents the development of competitive French research centers, In short, if you are not within the system, you have no chance to get in because you simply cannot know how to apply. There are in France four decent research places: universities (for traditional associate-professor and professor position), CNRS (full-research position in fundamental areas), INRIA (full research positions in Computer Science) and Grandes Ecoles (traditional professor positions but teaching only to small population of graduating students). Each of these places has its own recruitment process, with a lot of specific French acronyms and terms. They all require candidates to dive into websites with no guarantee that the English translated page is accurate and updated. For example, the recruitment at CNRS is based on the concept of Sections. Should you be able to know your Section, you might face some terrible issues like understanding this wonderful PDF. Universities have the worst recruitment process. Go to this portal if you are crazy enough to candidate, but please don't forget that if you want to apply for 2012 recruitment process, you first had to apply before Dec 31st 2011 in order to be authorized to actually apply four months later. No kidding. INRIA has the best recruitment process (with a LaTeX template for the application). Grandes Ecoles are autonomous small entities without much visibility. For the brand new Institut Mines-Telecom, the new recruitment website is still under construction, with a lot of infamous 404 pages and various links toward webpages in French.

The second drawback is the salary. In France, you cannot be a rich scientist. For CNRS and traditional university positions, it is even worse. A young scientist with a tenured position in an university cannot live decently at Paris. The salary (around 2000 euros monthly after tax) ruins your chances to find any apartment in Paris and nearby. INRIA and Grandes Ecoles are a bit more reasonable with monthly salary starting around 2500 euros. For example, after five years of experience, my salary is slightly higher than 3000 euros after tax. In all French cities (except Paris and Nice), such a salary allows a comfortable life, but it is still borderline for a family. The comparisons to other countries are not in favor of France. On the one hand, there are still many candidates despite the low salaries. On the other hand, best candidates, especially those who know the US system, are reluctant to apply.

This particular position at Telecom Bretagne. For young scientists with young (or upcoming) kids, Telecom Bretagne is an outstanding place, with a scenic oceanic coast, a kid-friendly climate (temperatures ranging from 5˚C to 20˚C), and cheap housing (a 4-bedroom house near the beach can be rented for less than 1000 euros per month). Brest is considered as one of the ugliest French city, but, with respect to world standard, it is still a nice lively city with an attractive homogeneous downtown. I would argue that Brest is typically a better place than the majority of cities in US, Germany or Japan.

Telecom Bretagne is one of these strange typically French "Grandes Ecoles". In short, the best French students go to Grandes Ecoles for graduating as engineers. Since Grandes Ecoles are dynamic small entities, specific partnerships with the best institutions all over the world have also been created in order to attract excellent abroad students (one third of students at Telecom Bretagne are not French). Hence, professors teach to small-size international population of smart students with whom it is possible to experience new pedagogical processes, including active learning and project-based education. However, these students will become engineers (the ratio of students willing to go to PhD is less than 0.1) and they are highly-demanding in average.

Fifteen years ago, Grandes Ecoles were only about teaching. The recent development of research activities explains the diversity of profiles in the so-called "departments". Professors with academic ambitions are free to develop their own research agenda, but they will have to find grants in order to hire PhD students and post-doc. Fortunately, the school has strong industrial relationships and good connections with major French research centers, therefore it is not that hard to find money if the research activity is within a hot area.

The day a rejected paper will end up in court

Conferences are no longer nice meetings among gentlemen who care about science. They are the center of a huge competition for thousands of scientists. An acceptance in a major conference can change the career of a scientist and the daily life of a laboratory. It is money and it is fame.

The goal of the peer-reviewing process has naturally shifted from validating the accuracy of a scientific work to selecting a subset of papers. The reviewers have mutated into hunters tracking any reason to crush a paper. All the gentle and polite manners from "guides of peer-reviewing" sound so XXth century. In computer sciences, the "hypercriticality" of scientists is well identified.

Unfortunately, all harsh reviewers are not gifted. Reasons for rejecting a paper frequently come from wrong beliefs, from major misunderstanding or from an obsession for a tiny flaw. These are negative badly-justified reviews, which sometimes make the authors speculate about a possible bias in the reviewing process: the reviewer was a competitor in the same domain, the reviewer was not an expert, the reviewer had personal grievances against the authors, the reviewer was a well-established scientist who does not want to see any newcomer out there...

What are the options for an angry author with a supposed unfairly-reviewed paper and its negative consequences (the money loss and the degraded career)? Let's call a lawyer.

The roles of every entity involved in the traditional academic process are well summarized in this nice paper. Sometimes, the editor of the venue describes the process in used. But the reality is that there is no well-defined rule, no engagement. In fact, it is written nowhere that every paper should be fairly reviewed. An author can only hope that the process is fair because the process is usually said to be fair. I have actually no idea what a lawyer could do versus a no-rule process.

I have always been astonished by Boeing (or Airbus) appealing a decision from an orderer when their bid is rejected. In my naive vision, how can anybody challenge the decision of a private company? When the orderer is the Air Force, there must be rules, for sure, but the idea is the same for a paper to, say, Infocom. But here is the point. There is no Infocom. More precisely, our beloved top-conferences might belong to anybody, including Infocom. I am not sure to know what a lawyer could do versus a no-entity.

Is is serious?

The rebuttal process is probably a good idea, which has the potential to reduce some unfair decisions. But it is still another prehistoric practice based on oral tradition and informal discussions between those who are supposed to be the owner of something that does not exist, except in author's mind. In medicine, some scientists have argued about the creation of a "supreme court" whose role would be to "judge the judges" (here and there). But the motivation is to prevent new scandals related to misbehaving editors. This court would deal with serious cases, not slightly unfair reviews. Maybe academics should look at what has been done in sports to legally organize the structures. Societies like IEEE or ACM could also consider to create a "mediator committee".

Otherwise a rejected paper will end up in court, then who knows who might be considered as responsible.