Cooperations, developments, projects: taming the complexity of research

In the last days I’ve been working round the clock — I’ve given my kNN search talk twice and several call-for-papers deadlines went by (or are coming), which meant that I was always in hurry for one reason or another.

This has been especially stressful, because for some time I had been getting everything done days ahead of the deadlines. But this month I went back to the classic regime of “crossing fingers and hitting the submit button at (literally) the last minute”.

As I move towards more complex research involving several labs, many students and audacious experimental designs I feel increasingly the need of using more formal management tools. But techniques created for Business (or even Engineering) do not seem to translate well to Academic research.

For example: I’ve tried to use Gantt charts in Microsoft Project to keep track of complex tasks and their dependencies. But I’ve found that as the work progresses, the list of tasks changes often and significantly, as some research directions reveal to be more fruitful than others. This ends up rendering the initial planning (and any chronogram based on it) useless.

Maybe is it the case of using an adapted “Spiral Model“, where risks are minimised from iteration to iteration, and creating detailed chronograms only for the lifetime of an iteration?  Or are there specific management models for research projects? How the most efficient R&D labs manage their projects? How to adapt their experience to Brazilian public research?

Unfortunately, so far, I seem to have more questions than answers. I’ve bought this interesting book “Managing Science: Management for R&D Laboratories“, which is biased towards Particle Physics labs, but has (hopefully) useful concepts for all areas of experimental research and will help me to get an initial handle on the subject.

2 thoughts on “Cooperations, developments, projects: taming the complexity of research

  1. Hi Eduardo,

    Hope things are going well during your trip in Europe. I’ve been for a while writing offline a comment about this post, but it grew too big, and needed some editing. Finally, being also traveling this week, I have been removed from my daily routine and could address some old items in my “to read” and “to write” lists, including this one. The comment is still too big, but I don’t have time to make it smaller!

    First, let me address the only disagreement I have with your post, which is the sentence: “But techniques created for Business (or even Engineering) do not seem to translate well to Academic research”. There is some misconception between “what really happens” and “what seems to happen” in the business world. A big part of any “business” is the creation of the image of a company as being both efficient and effective. Nothing better than to give the public and investors the image of a company that will work like a perfect clock, no matter the people that are working there. In reality, that doesn’t happen. It would be the same as expecting that “publishing laws” would mean “everyone obeys all the laws, all the time”. In the same way that people don’t dedicate their lives to continuously read all old and new laws and regulations, most processes in any company are written and edited regularly without most employees ever knowing about them. The academic world is then left with the frustration of trying to achieve an image that doesn’t match the reality. If a technique from the business world in not translating well into the academic/research world, most times you have to first doubt if it is really working in the business world.

    Having some experience in the academic world, a little more experience in the business world, and an increasing experience in the interface between the academic/research and the business worlds, I would have the following high-level project management “learned lessons” to share.

    1) The result is extremely important, but it may change. This is probably an area in which people in the academic world can teach a lot to people in the business world. One starts with the idea of researching or developing topic A, but that may fail, and yet produce brilliant results regarding topic B. That is acceptable in the academic world, and yet at times inflexibility causes disappointment and waste in the business world. Companies keep investing on projects, with an enormous sunk cost, just because they have the resources to do so, instead of reusing results in other areas, changing the project target or scope.

    2) Outsource what is not a “core research/academic result”. The resources are not lines in project files, but “alive” in the real world. Sorry to expose bad news, but people would be appalled to know that many projects are delayed or fail inside companies because of a single person getting unavailable, due to illness, holidays or simply “lack of motivation”. One can plan for an equipment to break, and have a backup available. It is far harder to plan for a replacement of a person with very specific skills, and years of experience in specific areas of knowledge, or tools. Mainly in the area of software projects, it is far easier to recover from a hardware/software failure than to recover, for example, from the unavailability of someone that should “simply” translate code from Matlab into a C++ module. That is where I believe that the academic world can learn a small lesson from the business world: parts of an academic/research project that are not the “core results” should be just considered tasks and possibly outsourced to an external provider. While nobody would think twice about outsourcing janitorial services in a research building, pride and ego get in the way of outsourcing a software component that is by no means core to a large research project. Typically a large number of small and specialized companies flourish around large corporations and more successful research centers. Yet, that mentality is still not so widespread, mainly in Brazil, and many companies insist on putting overqualified people doing trivial tasks, just because they can do it (creating awful scenarios that certainly contribute to unmotivated employees).

    3) Setting the right expectations is the best way to exceed those. Not everyone will be pleased all the time. One of the hardships of managing large projects is that different people and groups may have different goals. At times, it is easier to “sell a project” telling participants that by doing 20% of the total work they will get 80% of their personal or team goals fulfilled. Instead, it is common to see “experienced project managers” promising the unachievable 100% satisfaction for a “resource” doing only 20% of the project tasks. Worse is the assumption that having 20% of the total project tasks done independently by 5 different teams will automatically create a fully integrated and complete result. Failing to anticipate the integration tasks, and consequently who is responsible for them and an estimate of the time it will take for integration tests and modifications is another common mistake. Even if all such common scenarios can be planned for, it should be made clear at the beginning of a project that any “collaboration” always involves some “unanticipated frustrations”.

    4) After 50% of a project is considered done, establish the criteria to consider it 100% done, one way or another. Given what I shared in the first lesson, one of the main problems I see in many projects is trying to establish an exit criteria too early, and too precisely. Suppose your project is to design a software system to implement storage and retrieval of pictures in a large organization. A common mistake is including in the exit criteria precise sentences like “the web module should return answers in less than 5 seconds”. Later in the project the “web module” may be cut. The project gets then a module for mobile search of pictures. The exit criteria document keeps getting amended. Soon, the exit criteria is a Frankenstein document which nobody can ever use to decide objectively if the project succeeded or should be promptly cancelled because it will never fulfill its goals in reasonable time and cost. Despite my warning to wait a little until the project is better defined, ironically the best exit criteria documents I have ever seen are “project agnostic”. Someday I should just create an “Exit Criteria Template”. Such document basically establishes that the project will be evaluated after some time, and if it is not complete at a certain percentage it should be cancelled. Most other exit criteria documents that I’ve seen are not exactly exit criteria documents. They are either “high-level test goals” or simply a list of compromises to setup a project for success very close to the endgame, despite its many failures.

    • Hi Alisson,

      Thank you for your very elaborate comment, it has a lot of useful insights and I will certainly be reading and rereading it a few times. I particularly appreciate your honesty about the difference between how businesses actually work and the image of precision they try to project, because though this is something my perfectionist nature tends to forget.

      I also agree with all your points, in particular with the question of outsourcing (which is related, at least in the academic world, with the tendency of people “reinventing the wheel” instead of trying to make minor compromises to use the wealth of available existing code).

      * * *

      One other problem that I have witnessed a lot, of late, is what I would call “early engineering” (making a parallel to “early optimization”), in which researchers (often students) spend a huge amount of effort trying to give their projects “programming-in-the-large” qualities, while they are still in proof-of-concept stage.

      I encourage, of course, my students to code clearly and precisely (which means that they should try to make their programs bug-free and “human-readable”) but I also warn them to avoid “overengineering” their code for two reasons: first, our prototypes are not really intended for 100k LOC production projects, but for 5k LOC experimental projects. Second, because we lack real world experience and craftsmanship to do programming-in-the-large coding, the results of our efforts to do so are often very naïve. I dare to believe that one of the lessons that I have learned with you is that the transition from research prototype to engineering product takes much more effort than simply good intentions and good will.

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s