API management strategy
Organisations interested in building or managing APIs should put in place a management strategy. Guidance developed by the UK Government Central Digital & Data Office provides a helpful outline of the key areas that should be considered when making decisions about how to manage APIs. The following is a summary of this guidance.
Principles for the API strategy
Build good foundations
If you are creating APIs it is helpful to use an API gateway. API's benefit from the standardised tools an API gateway provides and means you don't have to develop and deploy those elements of functionality. Having an API gateway removes the necessity to build those tools again. It may however be difficult to take this approach if:
you already have APIs in production with their own security or monitoring
APIs have been developed using different methods and technologies
Establish the standardised patterns, processes and governance your organisation will use to build and deploy APIs. Focus on principles and frameworks rather than specific technologies, and consider how detailed and rigid these should be.
Build for the the future
Think about what your organisation’s future needs will be beyond the first few APIs. Maintain oversight of your APIs and a view around the maturity of the APIs in order to plan the resources required to manage them. Have an understanding of what stage each APIs is in their life cycle to know what possible issues are and what may be required to manage these.
Design API strategy around user needs
When making decisions about API management plan to meet both user and organisational needs and objectives. Work with development teams in your organisation work so your policies and processes are complementary and don't inhibit innovation. Undertake user research to gain insight into how teams are producing and consuming APIs and what technologies they are using or considering using.
Have defined roles for API management
Larger organisations may have multiple teams involved in building and maintaining APIs. The API management strategy can set out:
who owns different parts of the API life cycle
what skills or roles API teams should include
how ownership changes as an API goes into service
For example, a central team may run the API gateway and exercise control over service levels and capacity. This teams needs to interact with the design and delivery responsibilities of each API owner with issues such as versioning. Established structures for escalating issues will reduce duplication and confusion.
What API management strategy can help with
An API management strategy can:
provide visibility of your APIs, enabling people to discover them and potentially reuse
allow standardisation of common design patterns
help automate administrative and operational tasks
provide a central place for users to access API documentation and support
provide data and metrics that help understand API usage and performance
help implement and maintain good security and data protection practices
The API management strategy should cover the full life cycle from design to end-of-life (sunset)
Designing APIs
Establish design guidance to help maintain consistency across the APIs. It is easier to manage APIs, particularly at scale, that are consistent so use the same design principles and practices.
Provide a level of guidance that meets the size and structure of your organisation. For example, guidance might be different for an environment where API design follows strict standards, and an organisation using many different technologies and approaches.
Review and challenge assumptions
Often API strategy's are based on the first APIs built, with decisions and teams responding to specific problems. These teams may then be utilised in different scenarios that may not be as appropriate so it is important to test first assumptions.
User research can be used to test assumptions and define needs. Design research to help understand:
what technologies are in use, or might be in the future
the different levels of maturity of development teams in the organisation
whether the APIs produced are primarily internal or external facing, or a mixture
what service level agreements (SLAs) APIs should meet and whether different SLAs are appropriate for different use cases
Bring developers into the design process
API design guidance reduces the time and effort developers need to spend making decisions, by providing a framework for them to work from. It is helpful to explain design choices and this transparency increases developer trust and support for the process.
Use standards
Technology standards like OpenAPI can help increase confidence in software decisions. Standards can also provide credibility to your strategy. Use cases can help explain different API standards. As an example, you might require using REST for microservices and GraphQL for more data intensive processes.
Make the API design process easy to use
Setting out decisions in a central place helps development teams commence projects more easily. Keep these guidelines in a document store or repository that is easy for people to find and access.
API management tools can be used to store and share API specifications. Some API management also include a developer hub and an API gateway which can also be used as standalone design or governance modules.
Put API's through assurance and assessment
Assurance is a governance process to make sure APIs meet various organisational standards. Assurance should be seen as an enabler for teams to ensure they are building APIs that follow the design guidelines of your organisation or government.
The assurance assessment programme should be appropriate for the teams who need to use it. Make it clear what the benefits of going through assurance are so that teams see it as a worthwhile exercise to help them improve their APIs. If your assurance process is too rigid, or requires too much effort for not enough benefit, teams might try to avoid it.
Test and validate your APIs
Testing is essential to make sure the APIs are functioning properly, and is usually an automatic process to validate the code. Testing should be an ongoing process with regular validation of the API a part of ongoing monitoring.
Usually, testing is done by an automated tool against the specification and include checking:
specification against internal policies to make sure that APIs submitted to the developer portal are compliant with internal standards
API to make sure its responses match the specification and that deployed APIs are behaving correctly in production
Putting APIs into production
Deploying your API involves more than just pushing code to production. There are a number of things you should put in place to support the publishing process and to provide a good developer experience.
Document APIs
API documentation is the starting point for development teams using an API. Technical writers should be used from the start of the project. When documenting APIs it should include:
conceptual information to introduce the API
practical information such as a ‘getting started’ guide
detailed reference documentation
Use an API specification to generate and auto update reference documentation for the API as it is built and iterated.
Developer portal
A developer portal, also sometimes called a developer hub, is where external developers can access everything they need to set up and manage their integration with your API. This usually includes:
documentation for the API
developer registration and authorisation
self service tools for things like renewing API keys and changing their password
a test environment - for example a mock API, a ‘sandbox’ and test users
issue reporting and ticket support
Use the developer portal to gather internal metrics about your API programme. For example, it could be used to measure how quickly developers are able to set up a test version of an API. This is sometimes known as “time to live” or “time to 200” and is a useful metric to measure how easy the API is to integrate. It can also identify where there might be pain points.
Managing API gateways
The everyday operations of API service management are usually handled by an API gateway. Generally, an API gateway service acts as a proxy for APIs, and allows central services like access control and security to be provided for all the APIs using the service.
At a minimum, an API gateway should provide:
user and application authentication
rate limiting and throttling inbound requests
logging and reporting
Many organisations do not put an API gateway into production until they have several APIs in use. It is recommended that an API gateway is implemented at the very beginning of an API programme because this supports:
standardisation of central services such as network access, security and authentication, throttling and rate limiting, logging and monitoring
lower developer costs and reduced time to deployment,
required minimum standards of security, access control and monitoring, while providing an easy way for developers to meet them
Adding an API gateway reduces complexity in managing APIs and introduces a consistent front end for services. There are many different API gateway products available, both open source and proprietary. They will often include elements for a developer portal or a catalogue.
Making APIs easy to find
APIs should be visible, discoverable, up to date and in a central place. Use an API catalogue to help users discover APIs and understand how to use them. An API catalogue also helps keep an inventory of what APIs it manages.
Keep API discovery separate from your developer portal
Most API management tooling includes features for implementing an API gateway, the developer portal and an API catalogue, making it a convenient way to use one tool for all of these things. It is important however, to not rely solely on a developer portal to make your APIs discoverable.
There are cases where an API might not be appropriate to sit on the gateway or the developer portal, but should still be discoverable in a catalogue. This might be because the API is still in experimental stages, does not meet standards, or because the team is not ready to hand over management of the API.
Keeping the catalogue separate from the other parts of your API management tool allows discovery of all APIs without restricting them to those that meet certain criteria.
This helps:
promote an environment where innovation can happen in the open
avoid teams developing in silos to get around rules
improve transparency and reduce duplication of effort
Consider who will be looking for APIs
Most organisations have three levels of access for their APIs:
private - internal and kept entirely within the organisation
partner - shared externally with a limited set of trusted users and partners
public - open to external users either directly or through a self-service sign up platform
Consider how best to help each group of users find the APIs. This can be done with separate catalogues, but consider the cost and effort it would require to maintain these. Use access control to restrict visibility of APIs, or details of the APIs, to registered users at different levels of authentication.
Aim to expose as much detail as possible to all users. Even for sensitive data sets, expose basic details of an API with information about how to get access. This helps developers understand the value of the API and start interacting with it while they wait for access approval.
Check with internal security teams about what level of exposure is acceptable for each API. It may be helpful to review the metadata model for the government API catalogue. This may provide a basic discovery framework that does not increase the vulnerability of published resources.
Link the catalogue to the government API catalogue
An internal catalogue should update the government API catalogue, either programmatically or with manual submissions. This (government) catalogue is a centralised directory of government APIs, and any external APIs (whether inter-departmental, restricted or public) should be listed there.
Taking the API out of service
Plan how to take the APIs out of service when they are no longer needed (i.e. retirement, deprecation, sunsetting or decommissioning). Taking an API out of service suddenly or with poor support can significantly impact the services integrating that API. Plan for the retirement of APIs:
build trust with users - these are the external developers using the APIs
users plan for any changes they need to make to their services
protect the organisation’s credibility users
It might be useful to put together an API retirement workflow or checklist for the organisation to help teams follow a consistent process. Below is an example of what this might look like.
Use analytics to support the case for deprecation.
Publish a blog post to explain the reasons and offer alternatives where possible.
Add a deprecation notice to documentation with the date it will happen.
Disable sign ups in self-service to stop new users accessing the service.
Email subscribed users with the date of deprecation - as the date approaches, emails should get more frequent.
Use Sunset or Deprecated HTTP headers, with a link to the documentation and blog post.
Monitor how usage changes - make sure it’s dropping and contact any remaining active users.
Agree an acceptable number of active users at retirement.
Keep the API in retired status for a while - this could be months or years.
Publish the retirement policy somewhere external developers can see it such as developer hub or in API documentation. This will help reassure users of a good level of support throughout the API life cycle.
For more information see:
API technical and data standards - GOV.UK (www.gov.uk)