Methodology of development

Software development is a though problem since you can not really know with enough certainty what problems will arise and what is the better solution until the software is done.

This affects most projects, but this one is specially endangered since the complexity of the goal is very high and there are not many similar projects to take expertise from.

On the other hand, our goal is somewhat vague: Make real-time web application development easier, but how?

Here is described the methodology used in this project.

Propose an experiment

We first ask ourselves how could achieve our – somewhat vague – goal. We don’t know for sure, but thinking for a while a number of issues come to mind.

When we have several to choose, we must pick that one that we found most related to the success of the project and most risky. If the project can not succeed, the sooner we know, the better. On the other hand, if we get to address that issue successfully, we have not only made an important progress but also gained confidence for the rest of the project.

The first issue targeted by the project is data synchronization between a WebSocket server and a database. This issue is critical, not being able to synchronize data easily our project would be probably not much more helpful than a custom WebSocket server.

So the first experiment chosen for the project was: Make a WebSocket server that can synchronize data between clients and a database.

Draw first

Converting ideas into code is far from easy. Ideas hide many real world details that translate in many code changes, thought and redesign. Here is when UML comes to help.

Either you start coding from scratch or drawing diagrams you will find for sure many refinements needed to make the software work: There is a missing relationship, certain common functionality should be moved into a new base class, there are not enough data structures to clean subscriptions after clients disconnect...

All of these issues can be detected and fixed both in code and in diagrams, but fixing them before there is any involved code is much faster!

Being able to capture all the entities and relationships in visual diagrams is also a great form of documentation, specially in earlier phases of the project.

In order to use diagraming effectively, Snorky introduces its own idioms which extends the UML language in order to capture important implementation details. More information can be found in UML notation.

Code when you feel confident

Once you have drawn enough diagrams to hopefully cover your imminent implementation, and you have reviewed them enough to think you have not left anything important, you can start to code.

Open an editor, look at the diagrams, and choose a class with no dependencies. Code it and follow with the rest.

Chances are you forgot many details drawing the diagram. Do not worry, correct the mistakes, both in the code and the diagrams, and continue. Most fixes will be trivial since the most hard ones will have already been identified in sequence diagrams.

Throughly test

Coding the entire project and trying to debug all the problems later is just not productive. Debugging an entire project takes much time for each issue, and it’s very easy to miss details. Not something to repeat too many times without giving up.

Here is where unit testing comes to play. An automatic test suite is coded for each class. Unit tests must test only the functionality of a single class or at most very close relatives with no other relations.

In order to test relationships, mock classes will be used. They will only have the methods and properties really needed in the test case and often they will have no base classes. Duck typing as empowered by both Python and Javascript make this easy allowing mocks without need to define additional interfaces or empty stub methods.

Each test case is written just after the tested class. This approach is very effective because you get immediate feedback about the code you just wrote.

The test will be written taking into account all possible edge cases of method callings. Also, the original code will be reviewed for conditionals and exceptions and made sure they all are tested. This is known as gray-box testing, and is made much more easy by writing the tests just after the tested classes.

Often while writing the test some edge cases not treated in the source will be found. In some cases those mistakes will even trace back to the design. Since classes are coded and tested in an incremental fashion, the effort needed to fix the issues in this case is minimal.

Also, having an automated testing battery is invaluable when refactoring or making other design changes, specially in dynamic languages like those used in Snorky that lack type-checking.

As a conclusion, this testing methodology is a very good quality control measure, eliminating debug time, verifying both code and design and spotting problems then there is the least effort needed to fix them.

Reflex and find another experiment

Once the experiment is finished it is time to reflex about its outcomes. Does it work fine? Does the system have an easy enough API? Is it efficient enough? Maybe the design could have been done better? What is it missing?

Anyway, if it went well in the process we have learned it was feasible to address a certain issue and we even have a working implementation!

If the experiment failed it’s time to search for alternatives. Is the experiment issue that much important? Are there any alternative ways to deal with it?

At this point we return to the first step and search for another issue that needs addressing. The artifacts generated may be (and will often be) reused in later experiments. Integrating them into a single code base can also be an experiment.

Conclusions

The development methodology is a key to the success of the project. Being iterative and incremental we can develop with software of increasing complexity minimizing the risks involved. Testing often and everything allows us to write code more comfortably and assure the quality of the project from early design to the working implementation.

Table Of Contents

Previous topic

Historic context

Next topic

UML notation

This Page