Non-Functional requirements (NFRs) - Introduction
Consider the following scenarios:
2) You have built a mobile app on both iOS and Android as a fixed price project for a customer. But during UAT, you find out that both iOS and Android have gone through version updates/upgrades and now the mobile app behaves erratically for certain scenarios. Since it is a fixed price project, the customer is not ready to pay for changes (as you call them) or fixes (as termed by the customer).
3) You’ve built a middleware based on the micro-services principles using ASP.NET Web APIs for an Enterprise customer (let’s say a big bank in US). Now the security team of that Enterprise has found various issues in the code-base since it does not comply with OWASP top 10 security risks or vulnerabilities.
And the list can go on with multiple such real-life scenarios where you as a developer or as an architect or as a scrum master have faced that moment of truth - “Wish, I could have thought of this before!”.
All the examples above highlight the fact that the NFRs are the most critical yet generally ignored aspects of product/app development.
Functional Requirement (FR) vs Non-Functional Requirement (NFR)
NFRs are the requirements defined to cover the operational aspects of a product or an app while Functional requirement (FR) focus on the behavioral aspects.
Let’s say you are going to build a simple shopping cart app:
1) The app should display various products matching the search criteria is a functional requirement.
2) But, the app should display the products within 2 seconds from the time the user hits the “Search” button is a non-functional requirement.
Here are some other key differences between FRs and NFRs:
1) NFRs describe the overall experience of using a product or app while FRs obviously are focused on a specific set of functionalities.
2) They are typically stated as implicit requirements (i.e. always expected than stated).
3) FRs can be captured at a user story, use case or a function point level. NFRs usually exist across the product (and also for the lifetime of the product). For example, Usability or UX is a product-wide or app-wide requirement and should be considered even in product updates/upgrades/enhancements.
4) NFRs are traced across various modules of a product/app like front-end, middleware or databases, etc.
5) NFRs are often classified as a part of software quality assurance (SQA) process and rightly so.
In short, the FRs define “what” and the NFRs define “how”.
From an end-user perspective, both are important and they should work hand-in-hand.
The Curious Case of NFRs
Now let’s look at NFRs from different perspectives.
SDLC & NFRs
9 out of 10 projects that get executed today are Agile or Scrum based.
How many of them actually follow Agile processes? Or are just called as Agile for the brand value is a totally different topic. We’ll keep that for a separate discussion some other time.
Irrespective of the process you follow for developing your product or app, NFRs have to be considered from Day 1. However, the kind of process you follow will drive the way you handle NFRs.
Let’s see how:
In Agile development, you work in a sprint-model with theme > epic > story > task breakdown.
So, when you are breaking down the functionalities, you need to also ensure that you plan for implementation of specific NFRs as a part of sprint-model.
UX/usability would come up first in terms of priority of NFRs; however, scalability, performance, security, browser compatibility will follow. For any epic/story/task, be mindful of the NFRs and come up with task breakdown and estimates.
Here are various possible ways to handle this aspect in agile development:
1) Create the acceptance criteria for story points that covers NFRs. So, for a specific story, you can add an acceptance criterion that says the app should function well for 50 concurrent users. So when the developer has implemented specific stories, he/she needs to also ensure that the story passes this acceptance criteria.
2) Another way is to have the DoD (definition of done) cater to specific aspects of NFR. For example, once you define UX guidelines for a project, the DoD should mandate that UX guidelines are followed in developing any new page or UI screen. This needs to be mandated for developers and also testers.
3) A more suitable and process-oriented way could be creating user stories for NFRs as well. So you can follow the typical “As an X, I want Y, so that Z happens” format but catering to NFRs. So, a story catering to high availability would be – as an end- user of the site, I want the site to be accessible 99.9% of the times so I could perform various operations seamlessly without down-time.
These are various ways of incorporating NFRs into implementation process from early on. The model or approach you would like to follow can be any of these as long as it works for the team and, of course, NFRs are a part of the delivery process.
Since the waterfall model itself forces you to do a thorough analysis of requirements followed by detailed design including logical/physical architecture design, chances are high that the NFRs are analyzed and detailed out well enough in analysis and design phase.
However, this needs to be validated by taking a sign off from the customer on things like response times, performance benchmarks etc. Else you are again falling into a trap wherein the implicit requirements or expectations from the customer (or from customers of customer or end-users) would differ greatly from what you have coded for.
Team Roles & NFRs
It is evident that NFRs are important and critical for any and every member of the team – be it scrum master, developer, QA, release manager or a support engineer. In each role however, the scope of understanding would differ:
1) For a Product Owner it is critical to have the overall picture clear in terms of goals or vision for building a product or software. After this, with help from software architect, the NFRs need to be broken down into stories or acceptance criteria (as explained earlier) and the sprints have to be executed accordingly.
2) For a Scrum Master, it is critical to understand the cost aspects of not handling NFRs in due course of the development process. What is also important for Scrum Master is to know as to how to incrementally get the NFRs implemented and delivered through the agile/sprint-based process.
3) For a business analyst (or a product owner), the end-user perspective of NFRs is more important. BA should be able to elicit the NFRs clearly so that the dev team can understand. BA/PO should be able to understand the implicit requirements by talking to end-users.
4) For a developer or QA engineer, it is important to understand the impact and relevance of each NFR on the specific functional requirement that he or she is implementing (i.e. either writing the code or test cases or automating them). And this is irrespective of years of experience the person has. The habit of thinking about NFRs needs to be inculcated from day one of their careers.
With NFR affecting so many variables in the equation, how do you ensure that you’ve covered them all?
One solution is to use requirement traceability matrix to track NFRs across various SDLC phases and across various product releases. So for a specific NFR – say security – you would be able to trace it well across various stages of product development.
Meet the Supporting Cast of NFR
By now you must have realized that NFRs are essentially those important cogs in the wheel that appear to be trivial but have a bigger impact if ignored, just like a supporting actor in the movie.
You always focus on the key actors but their performance is complemented well by supporting actors. If they don’t play their part well, the performance of main actors may not have the right impact.
NFRs can be your best supporting cast or best villainous performers depending on how they are envisioned, implemented and managed.
We’ll be focusing on the following supporting actors:
- High Availability
- Internationalization and Localization
- Browser Compatibility
Let’s understand them one by one in depth.
Scalability is the ability of a product or an app to scale up (or down as well) to fulfil the changes in app usage demands.
When you design a product or an app, have some idea upfront about the number of users that would use the product or the app. If you don’t design for that, then the application would not perform well.
Not all the users (i.e. total number of users) use the application at a time. Hence you would also need to know how many users will use the product/app at a given time, also called as concurrent users.
Concurrent users are generally 20 to 30% percent of total user base. However, it can go up to 100% for specific events – for example, your ecommerce product has launched a Christmas sale. Or your football match portal has gone online for selling tickets for a match and so on.
While total and concurrent users are the right metrics to track, you should not forget the time-factor as well. To handle such events, your app should be able to scale up/down based on needs.
Usually there are two kinds of scaling requirements:
1) Vertical (i.e. scaling up): When you are adding more CPUs or RAM to existing infrastructure, then you are essentially scaling up the infrastructure or vertically scaling it.
2) Horizontal (i.e. scaling out): When you are adding more servers to the same infrastructure, then you are scaling out the same infrastructure.
Following is a pictorial representation of both approaches:
Figure 1 : Scale up vs Scale out
So, enhancing the capability of same node is scaling up or vertical scaling, whereas, adding a new node altogether is scaling out or horizontal scaling.
Most cloud providers like Azure or AWS “auto-scale” your infrastructure and have required interface to allow you to configure these things. Here are some best practices for Azure auto-scaling.
For an on-premise deployment though, you have to ensure that you understand these nuances well enough and are able to handle the scalability well. In this case, the capacity planning activity is very important. In this activity, you will need to understand the overall footprint of the product or app that you are building and then plan for hardware resources like servers, memory, CPU, database servers, web servers etc.
An often-ignored aspect in scaling is the role of Software Architecture.
There are umpteen instances wherein the software scaling is not thought through during the design phase. You may scale out or scale up but if your cool data grid on a single page cannot load up in first place because it loaded data for 50,000 rows during page-load, hardware scaling will not help.
In essence, while you have the hardware to rely on for scaling, it is important that the software is written well in such a way that it produces an optimal performance in the first place.
Some key aspects to consider to make this happen are:
1) Software scaling usually is a function of the underlying technology/framework you are using. If that can scale, your app can scale. If your framework is good at doing multiple things at once (not only multi-threading), then you are good. Most modern-day technologies fulfill this aspect. However, if you are using the features of technology or framework in the wrong way, it may not work.
2) If you are using any legacy component as DLL or an API, then you need to ensure that that does not become a bottleneck. Think of some critical C++ library that your web app is using. The library also needs to scale well in order for your app to perform well.
3) If you are using a load balancer in the physical topology, then you would be able to also gracefully handle the architectural scaling problems.
4) Having right data caching policies at the web server, database level and also at data level could help in handling the scaling aspects well. However, this needs to be analyzed on a case-by-case basis.
5) In case of a database, use and define Views where needed. Also use proper indexing to avoid specific issues.
6) Session management needs to be used effectively to ensure that application performance is optimal
Even after taking some preemptive steps you would have to also take care of some corrective steps when you encounter issues after deployment.
Performance of a product or an app defines how a product/app is performing or behaving as compared to its expected behavior.
Following are the key metrics tracked in the context of performance:
1) Response times: This is a classic metric used for analyzing the performance of a product or an app. It defines how fast or slow your product or app is when a user tries to use it.
An associated parameter is the latency in responses. So, some amount of latency (i.e. delay) might be acceptable to users but it cannot go beyond a threshold. That threshold is response time.
You would typically define the response time per screen or for critical screens. Or you can define the requirement as 80% of the screens would have a response time of less than 3 seconds.
Latency is attributed to resources used while providing response to user (hardware, communication protocol, data center etc.), whereas response time includes latency and the processing time (i.e. to build a response for a specific page).
2) Throughput: This one typically refers to the number of transactions that your app or product can handle at a time. It could be specific number of user requests and/or database transactions or API calls and so on. You would define the throughput goals as a part of requirement analysis phase and would have performance/load testing phase to validate the product or app’s performance.
Strategy for Performance Analysis
“In god we trust, all others bring data” – W. Edwards Deming
This quote exemplifies the strategy to be followed for performance analysis and tuning of your product or app. As long as you have data that depicts the issues in the performance of software, you can analyze and fix it. And as long as you have data that proves that your product or app is performing well on given KPIs, then you can be rest assured that all is well.
The performance aspect of a product or an app is handled in two ways:
1) You need to first ensure that your architecture or design is fool-proof enough to handle the performance metrics for which it has been designed. For this, the analysis phase needs to focus well on the desired performance benchmarks for response time, throughput etc.
2) Once done, in the system testing phase, these aspects are required to be verified and validated through test automation tools like JMeter.
However, it does not end here. Many times, when the product or apps go live or the functionalities get changed/added, the performance of the product or app may deteriorate.
In that case, you need to use the following approach:
After running this process multiple times, there are still situations when your product or app exhibits a random behavior and it is difficult to pin-point the cause of the issue. In that case, keep on using the trial-and-error methodologies to single out the problem.
However, there is no secret sauce or a well-defined formula for fixing performance problems. It’s like an adventure park ride. You have to experience it to enjoy the thrill.
Let’s look at a few tools used for performance testing:
There are quite a few automation tools available and this list can be big but I have provided only a sampling of some popular tools. Please note that there are various UI/automation testing tools like Selenium that I have not mentioned here since we are more focused on performance testing aspects.
High Availability (HA)
If you need your web app to be available for serving users at all times (or almost at all times), then you need to design it for high availability. It is often stated in % terms like HA for a web app is 99%. What this means is that in a year, the system will be guaranteed to be available for 361.35 days (i.e. 99% of the year).
This is an indicative architecture diagram for a web app deployed in cloud (AWS/Azure or any other cloud provider) that has the following components:
1) Load balancer for balancing the varied load that the site would get.
2) App server that would serve user requests. It would host the UI and business logic layers.
3) Database servers with replication.
Figure 2 : 3-Tier HA architecture
Following are the key design principles followed in the HR architecture shown in Figure 2:
1) Design for failure: The architecture has redundancy built into it so that the failure of a server/resource does not impact operations. App servers and DB servers take care of that aspect. The load balancer itself can be a single point of failure. But this is already handled by the respective cloud providing solutions (e.g. EC2 based elastic load balancer in AWS).
2) Availability zones: Azure/AWS provide HA through the use of availability zones. Azure availability zones are unique physical locations providing redundancy within specific Azure regions
3) Auto-scaling: The cloud provider already has ability to auto-scale the infrastructure, and it is used while deploying the infrastructure.
In a nutshell, HA is a critical NFR that needs to be thought through during the design and analysis phase. It certainly cannot be an after-thought.
Security has become an NFR of paramount importance of late due to the amount of data collected by various products/apps and the vulnerabilities that are being exploited by hackers for theft of data.
OWASP (Open Web Application Security Project) is an online community that focuses on improving the security aspects of web apps. It publishes a guidance every year about top 10 security vulnerabilities and how to handle them in your apps.
The following table summarizes top 10 OSWAP security vulnerability areas for 2017.
Certain aspects of security assessment can also be carried out using various threat modelling tools available. Microsoft has a threat modelling tool available that can be used for performing a detailed analysis of possible threats during design phase itself. Plus there are many more such tools available as well.
“A user interface is like a joke. If you have to explain it, it is not that good.”
It is clearly evident that usability of UX is vital for your product/app to acceptable to users. Here is an in-depth article for you to understand the Usability or UX aspect well enough.
Internationalization and Localization
This is yet another NFR that surfaces fairly later in the game and would impact the overall execution in terms of additional testing efforts. Hence it is prudent to consider this aspect during the design phase.
Before we move on, you need to understand the basic difference between internationalization and localization. It is as follows:
1) Internationalization (often stated as i18n) is a process by which you enable your application to be used in various local languages. You follow various processes like using resource bundle etc. so that your app can be used in languages like German, French, Hindi, Chinese and so on.
2) Localization (stated as l10n) is making your product or app adaptable to a locale or a local language.
3) So i18n enables your app for l10n. Got it?
If your product or app will support various spoken languages like German, French, Chinese etc., then your UI should support it. Have the UI strings or labels on your screen to appear in these languages. What this means is:
1) Have the right translation resource bundles in your code projects.
2) Factor in additional testing efforts for the entire UI for the languages supported. Remember that the entire app needs to be tested for multiple languages.
3) Ensure that the UI/screens look consistent in terms of white spaces. For example, the German translation of a word occupies longer space as compared to English one.
4) For Arabic language, the screen should display from the right to left for all strings. So your app will have a different look altogether for Arabic UI.
5) Be mindful of numbering formats followed. In French, comma is used as a decimal point. A French-speaking user would need the commas for decimal points whereas an English-speaking user would need dots!
6) UI text or labels are fine but what happens to data on the UI? If you have entered a value in French with a comma as a decimal point – say 9,92 , what do you store it as in the database – 9.92 or 9.92? And how would you display it in a report that gets generated later? All these questions have to be asked and answered during analysis phase.
Remember that friend of yours who likes Firefox over Chrome and you’ve fought with him about how Chrome is better than Firefox?
Aside from a nice coffee-table debate, browser compatibility becomes a key aspect in product or app development. Have you not seen banners like “best viewed in Chrome XXX”?
Well, that explains it!
It is not only about the browser you use. It is also about the form factor.
If you have designed a web app or a hybrid app that a user would use on the web or on a mobile/smartphone, then you need to ensure that it renders well on both form factors. If your UI is responsive i.e. it adapts to the various form factors, then you have won half the battle. That is why it is critical to get this NFR sorted out in design (that too UI/UX design) phase itself.
Testing for Browser Compatibility
With so many browsers and various form factors to handle, are you worried how can you test for browser compatibility?
Well, there is a tool for that! Following is a quick summary of such tools available:
Chrome Developer Tools: I’m sure you have already used this. If you go to Chrome > Developer Mode, you can see the DotNetCurry (DNC) website:
Figure 3: Various Device Views in Chrome Developer Tools.
The options shown in the highlighted boxes in Figure 3 will allow you toggle the device/web view, and various device views. This way, you can quickly check if your web app would look stunning or not so stunning for different form factors.
If you want thoroughly test your app end-to-end for browser compatiblity, then you have various tools that can be used.
Here’s a quick summary of some tools:
Since there is so much user data collected and processed by various products or apps we use, the privacy and control of data by the creators of apps/products has become very critical. And that has led to a very important NFR surfacing up–the compliance requirements.
Why would this matter to you as a developer?
Well think of a situation where your product or app has been GA’ed (a.k.a. launched or General Availability) and now a specific customer wants to ensure that there are enough audit trails or audit logs to comply with regulatory requirements of a government body.
This means that you have to go to all those places in code where you are changing data (almost everywhere!) and add logs or trails.
It does not end here; you have to have a mechanism to generate reports as well. If you have not catered for any of this, this implicit new requirement is touching almost all parts of your product or app. How is that?
The following table summarizes such compliance standards you need to know of (the list is not exhaustive):
This* > https://github.com/ewoutkramer/fhir-net-api
If you are building a consumer app, then you need to be mindful of the possibility of your app being used by someone who is hearing impaired or is blind or has some physical disability.
And this aspect needs to be analyzed during the requirement phase itself. If you intend to build a web application or any application for Federal agencies in USA, then you need to comply with Section 508 or Section 508.gov that covers the accessibility aspect well enough.
There are similar such acts in other countries (viz. in Australia Disability Discrimination Act) wherein the government mandates to have public facing apps to comply with accessibility.
The W3C consortium has Web Content Accessibility Guidelines (WCAG) detailing out various aspects of accessibility.
Here is a tool that you can use to ensure that your ASP.NET app is complying with accessibility requirements.
Have you seen the batman movie, The Dark Knight?
It is one of top ten highest grossing movies as per this link. Ok, why am I bringing it up here? Just bear with me for a while.
Assuming that you have seen the movie (and if you have not! go see it!), do you think that the joker aka Heath Ledger is the supporting actor or the main actor?
Well, he won the Oscar for the best supporting actor that year! However you couldn’t think of the movie without him. That would be an example of how the supporting actor has stolen the show! So you never know when your supporting cast could take the center stage and make or break the whole thing.
NFRs are like that.
If you pay enough attention to them, they’ll win the show for you.
So next time when someone argues with you about the importance of NFRs, you can, to quote the joker, “Smile – because it confuses people!”. Because only you would know how important NFRs are!
This article was reviewed by Subodh Sohoni.