First posted Sep 29, 2024
This situation is practically inevitable in software development. You are tasked with extending code that someone else wrote. This can happen if you are the new person on an existing project. It can be because you decided to contribute to an open source project. It can be because you were assigned to extend some software that a former employee wrote, and no one else knows. The open source ecosystem has generated a massive amount of code that no one is working on, or intending to work on. Here are some suggestions for dealing with these situations.
Your first inclination will be to just wade in and make the needed change two minutes later. If the code is small enough and simple enough that you can do that, give it a try.
Most often, two minutes later you realize that this is a big, complicated project with dozens of source code files. You don’t know where to make the modification, or how to to make it. This next part is counter-intuitive, but once you have experienced it, you will probably agree. The fastest way to make a small change to a big code base is to approach it like you expect to spend the next decade of your life modifying the software.
Step 1) Don’t do it.
If someone is still writing, maintaining, or updating the code, it may be better to submit a bug report or feature request and let those people handle it. That leaves you with a few options like waiting for them to make those updates, using a different program, or writing your own.
Step 2) Be social
If you found a bug, see if it is already documented. If not, file a bug report.
If there is an online group for developers of the code, join it, and say hi. Saying something about how you plan to work on X part of the code could get useful suggestions, requests to look at a certain bug, or thanks. It might also result in someone saying not to do that since George is working on exactly this, is 95% done, and it’s best just to wait. In the world of branchy development, there can be a lot of code out there that just hasn’t yet been merged in with the development tip. If nothing else, they will know who you are when you submit a pull request, so that will go more smoothly.
A big coding project will have its own culture. Maybe you need a blessing from the head guy before working on something, or they will reject your pull requests. Maybe they want you to do some documentation or bug fixes before they will trust you to add new functionality. Maybe you need to be a student of one of the main developers, or collaborate with their academic research group. Maybe they need you to sign some legal agreement. Asking questions and feeling out the culture will be far less frustrating than doing some work that no one will incorporate into the code base.
If it was developed by one person, try to contact them. Many people want to have their projects turn into community project, but just don’t know how to make that happen.
If it is open source and the original developers can’t be contacted, you can start a new, derivative project. However, this is best left as a last resort.
Step 3) Read the documentation
Read the user documentation so you have a view of how the software is intended to be used. If there isn’t one, make one as you back engineering the code.
We’ve seen software with no documentation at all, software with minimal user documentation and no developer guide, software where all of the documentation is for developers and there is no good user guide, software with a massive amount of documentation, and everything in between.
Step 4) Make a functional test suite
Functional tests are tests you can make without examining the code. Have a whole set of input files and something that checks for correct output. If it is graphical software, this may be just a written procedure saying to click on this button, then showing a screen shot of what you should see. Include some atypical or corner cases.
You would be surprised how often you find bugs in code without ever looking at the code. It’s better to know about them now than to wade into modifying a code assuming it was perfect before you started.
Step 5) Read the developer’s guide
The developer’s guide may be called a programmers guide. 99.9% of the time, the former writers of the software didn’t write a developer’s guide.
Step 6) Write or update the developer’s guide.
Write a developer’s guide, or update the existing one. Being an old vi jockey, mine are usually just a text file added into the source code tree. In recent years, wikis are more common, particularly for large projects, or project you want to evolve into a large community developed code. Some things I like to put in this document.
Use cases
- What was the software designed to do.
- Notes about what was intentionally out of scope are also helpful.
- Is it intended to be all graphical, no GUI, both?
- Is it designed to be single user or multiple user?
Assumptions and limitations
- Is the program limited to only handling certain types of problems, input data, a maximum size problem, etc.
- Is it parallelized or ported to GPUs?
Architecture description
- Is it one big program or multiple little programs?
- Does it rely on a common data format internally?
- Is it designed to be interoperable with other software?
- Is the input philosophy ease of use or a powerful set of features?
- Are there coding trade offs, like performance at the expense of code simplicity, or code readability at the expense of performance.
- Input and output file formats
- Algorithms or design patterns
- Internal data structures
Procedures to compile, build, or install the software
- You may have to configure the environment in your computer in a specific way to compile it, so document that too.
If it has a test suite, run it.
- Document how to run the test suite.
- Document what the output should look like if everything is working correctly.
How to run it
- A procedure to run a small example outside of the automated test suite.
A list of the directories the project is organized into
- This might be combined with the source code list.
A list of the source code files
- Include a one sentence description of what is in them.
- Most of the automated utilities that do something like this give way more information than the birds eye view I’m looking for. I’ve never found a great utility for making this, so I usually build this by hand. It might include one line for each object class with its one sentence description.
Maybe use automated utilities
- Some people like the automated utilities that generate a list of all of the functions with their arguments.
- That may be supplemental information that is not part of your developers guide.
A call graph
- There are lots of great utilities for this including CodeViz, Doxygen, piping data from clang to opt to dot, CppDepend, and cflow. For a recent project, the simple cflow utility worked well.
A list of dependencies
- Today it is rare to find code that is 100% self contained.
- It may be calling certain math libraries, frameworks, APIs, python libraries, etc.
Theory notes
- If you find anything in the documentation, comments, or otherwise indicating that the software uses the X algorithm, or is described in the Y academic publication, include that in your developer’s guide.
- Some scientific software has a whole theory manual.
Style guide
- If the project has a style guide, point to it, and make sure everything in your documentation is consistent with that.
- If it doesn’t have a style guide, document the original code style.
- If you find style is bad or completely lacking, start considering how to get it from what it is to what you think it should be.
A todo list
- As you are getting comfortable with the code, you are going to find a dozen things you would like to change about it. Start making a list.
- This goes in the bug database if it has one. Otherwise it might go in the programmers guide.
Step 7) Beautify the code
No, we aren’t nominating the code for sainthood. However, there may be plenty you can, want to, and should do to make the source code look nicer without changing any functionality.
If the project has a style guide, follow it. Often considerable improvement can be made by following the style guide better, or within the constraints of the style guide. If the project doesn’t have a style guide but should, maybe it is your time to champion that cause. You will find a few people who are against any change or anyone telling them how to do their work, but more will be wanting the same thing that you are suggesting.
The developer’s guide you made above should get you to the point of figuring out where in the code you want to make the modifications for the task at hand. As you start looking at that section of code in detail, consider doing some of the following.
- Make unit tests. This is often done in parallel with all of the steps below.
- If the project has a linter, that is a program that automatically updates the code style. Use it frequently and stay within its constraints. Or start a discussion about updating the linter’s rule.
- Likewise the project may have coding metrics scripts that don’t change the code, but do kick out a report of why a given section of code doesn’t conform to their style guides.
- Add comments. You can never have too many.
- Rename variables. How many times have you seen a variable named “distance” with no indication of whether it is in millimeters or miles. Once you’ve spent an hour figuring that out, rename the variable something like “distanceKm”.
- If you have a favorite variable naming convention, implement it. Hungarian naming is designed for massive projects with hundreds of software developers. Many people use some subset of Hungarian naming.
- Clean up indents. Add them, change tabs to two spaces, whatever makes it feel better to you.
- Add blank lines.
- Add error traps. Lots of code makes undocumented assumptions about the input file format, function arguments, etc. The code that reads input data should have every error trap you can devise to report missing input data, invalid data, etc. Likewise if a function will explode if handed an argument that is zero or negative, add an error trap. In the language of computer science, an error is something that makes it completely impossible for the software to work. As such, all error traps should print a useful message and have an exit statement immediately after the error message. There is an art form to writing useful error messages. Which would you rather see as a user of the code “X is negative” or “Error: function my_sqrt() can not act on a negative number”? When you have a hundred thousand lines of debugging output, you will appreciate being able to grep on “Error”.
- Add warning messages. In the language of computer science, a warning is something that might be safe to ignore in certain circumstances, but is concerning enough that it would be useful for the user or developer to know about. Most programs have many error traps and a few warning messages.
Step 8) Modify the code
You’ve made it! You are finally to the point where you can start fixing bugs, and adding new functionality. If you found bugs in the course of the steps above, fixing bugs first is often a wise choice.
If the code is from a big project that has style guides, linters, etc. work within their style. If you don’t understand that style, learn to work with it before you write. Surprisingly, even the most rigidly managed coding projects still have a lot of room for personal style, and a lot of room to code well or poorly within their guidelines.
Make small changes, especially at first. You will be surprised how often a little change in one part of the code has a major effect on something somewhere else. Stick to modifying the part of the code you are working on and leave the rest alone. Don’t take on a major refactoring until you feel like you know the code inside and out. Sometimes convoluted, complex code was made that way because it is massively more efficient than a more understandable design. A good order for making changes is;
- Bug fixes.
- Tiny feature additions
- Small style changes
- More significant functionality additions
- Bigger changes to look, feel, style
- Major rewrites (maybe never)
Last, but not least, add it to your resume.