Learning From Forensic Science: The Fingerprints Of Developers
The World Of Forensic Science
Has the latest software build you received got the fingerprints of developers all over it? I was taking part in a MOOC with FutureLearn the other month on an introduction to forensic science. While you have probably not been working on software that has been murdered like in TV shows such as CSI and NCIS? However some testers may disagree with that statement (of course I would never describe the developers at Impero in that way). However we can learn a lot from the world of forensic science.
Television shows often glamorise the job of forensic science the character Abbey out of the hit TV NCIS does offer a glimpse into the life of crime scene investigation. When NCIS the Regional Forensic Lab director Dawn Sorenson (Abby’s real life counterpart) had a tour of the studio that makes NCIS she told Perrette, “You make us all look good, so we’re grateful. While what is on television vs real life is of course very different. I am sure it wouldn’t surprise you to find that not all forensic work gets done in 10 minute montages like on CSI. However the skills and frameworks used in real crime scene investigation have a practical use in the world of software testing.
TV often shows forensic science to be exact science and this often can misinform jurors creating something called the CSI Effect. Most forensic science is a human analysing the results. Take fingermarks the television tends to show a computer getting an exact match between a fingermark left at the scene of a crime and a fingerprint of the suspect, However the reality it is a judgement call of a technician based on their experience using a framework for guidance.
Lets take a look at one of those frameworks which is fingermark identification (fingerprint is a known for example a fingerprint of a suspect where fingermarks are left at a crime scene) which uses an international procedure called ACE-V.
The first point of call is for a forensic technician who looks at the fingermark taken from the crime scene and looks for the three main patterns — loops, whorls and arches. This is a high-level view which they use to classify the prints or to rapidly exclude suspects.
At this stage the technician is comparing it to a reference finger print using the three different levels (loops, whorls and arches) looking for differences or similarities to the reference print.
Next the technician then evaluates the information and decides an outcome based on one of three choices; the fingermark matches, it does not match or is at least one major difference that cannot be explained by other factors such as smearing; this means that the mark is not clear enough for the examiner to check.
Then the last step is to verify the results. A second technician verifies the work of the first technician to make sure that they get the same result. In fact failures in fingermark identification in the past have often been down to the verification stage being conducted by the same person as the first three steps.
ACE-V Procedures In Software Testing
If we are to transfer this framework into the world of software testing then you would start with a build that is ready for testing.
You look at the software with a high level overview. Does it work on a basic level while the obvious answer is does it pass the smoke tests are there any other tests that we need to include?
At this stage the person testing the software is looking to compare it against some defined standards which in the world of software testing this could be such as functional requirements documents or a domain expert advising that all software of this type have these common features. We compare it to our previous knowledge such as trying edge cases on different fields or that certain characters tend to break the input boxes. We then check its usability against our own knowledge of the customer and their technical level. We then compare the software against its intended context.
In this stage we make some judgement calls on the software based on what we found in the comparison stage. The outcomes here are; Does the software contain a huge risk associated with moving on to the next stage or not? Can the risk be migrated in some way which means that the software can continue or is the risk a low-level risk?
However the evaluation stage is often where we stop as testers. We have made our evaluation and now it is up to the business to decide based on your risk assessment. The question is should we be verifying? Should another tester look at the module or software, looking at our outcomes and seeing if they conclude the same. Interestingly in the past when errors where made with fingermark verification a number of times it was down to people skipping this stage due to urgency. While with this stage we do have to careful we don’t duplicate the work just done. I do think if you have worked on a module getting a second person to look at it before you sign the module off is worth the investment.
As I previously said there is a lot we can learn from forensics science disciplinary and ACE-V framework just scratches the surface of the potential it can offer. Bringing the ACE-V framework into software testing reminds us that after we looked at a module or some software getting someone else within the team to look over it even if it is for a quick ‘once over’ will help us create a more stable software.