1. Background introduction
The main issue discussed in this article is to use Amazon API Gateway to complete cross-account or cross-network API integration in a private network environment through Private Integration and Private API. The API management platform will be designed in a separate account (Amazon Cloud Technology provides a multi-tenant environment), because the existence of different business systems objectively needs to be integrated with the API management platform, so the API management platform is likely to be connected Different network environments may be other regions provided by Amazon Cloud Technology, offline environments, or other cloud environments.
| The Amazon cloud technology developer community provides developers with global development technology resources. There are technical documents, development cases, technical columns, training videos, activities and competitions, etc. Help Chinese developers connect with the world’s most cutting-edge technologies, ideas, and projects, and recommend outstanding Chinese developers or technologies to the global cloud community. If you haven’t paid attention/favorite yet, please don’t rush over when you see this, click here to make it your technical treasure house! |
Before discussing the topic, let’s talk about the API management platform. APIs are a typical part of modern application development. They enable applications, systems, and services to communicate effectively, breaking down silos of underlying technologies. To get the most out of these APIs in an efficient and scalable manner, API management is a must. Leveraging an API management platform enables businesses to plan, design, test, publish, operate, secure, and version control their APIs in a safe, easy, and efficient manner, thereby reaping the maximum potential of the data and services the business can provide.
Amazon API Gateway is a fully managed service that makes it easy for developers to create, publish, maintain, monitor, and secure APIs of any scale. APIs act as the application’s front door, allowing access to data, business logic, or functionality from your backend services. Using API Gateway, you can create RESTful APIs, Http APIs, and WebSocket APIs for real-time two-way communication applications. API Gateway supports containerized and serverless workloads, as well as web applications.
API Gateway manages all tasks involved in accepting and processing thousands of concurrent API calls, including traffic management, CORS support, authorization and access control, throttling, monitoring, and API version management. API Gateway has no minimum fees or startup costs. You only pay for the API calls you receive and the amount of data you transfer out.
We have observed that different customers are evaluating or deploying API management platforms. Amazon API Gateway, as a cloud-native serverless architecture API management platform, will be able to meet your API management needs very well.
In addition, applications implemented based on the OpenAPI interface specification can automatically generate documentation for methods, parameters, and models. This helps keep documentation, client libraries, and source code in sync. And the OpenAPI specification is language-neutral. Through OpenAPI’s declarative resource specification, clients can understand and consume services without knowing the server’s implementation or having access to server code. API Gateway also supports OpenAPI v2.0 and OpenAPI v3.0 specifications.
2. Concept introduction
When you are using API Gateway, there are some key concepts and definitions, which are the prerequisites for achieving the main functions of API Gateway.
Integration (integration), this is the core concept of API Gateway to realize its functions. Whether it is Restful API, HTTP API or WebSocket API, the interface for access exposed by API Gateway is connected with the specific implementation of the backend through the key action of Integration. The backend here may be a Lambda function, other Amazon services, or other HTTP resources, or private resources placed in a VPC or resources in the internal network. However, the REST API is composed of resources and methods; the HTTP API is composed of routes and resources; the WebSocket API is also composed of routes and routing keys;
A REST API consists of resources and methods. A resource is a logical entity, and applications can access resources through resource paths. A method corresponds to a REST API request submitted by a user of your API and the corresponding response returned to that user.
The HTTP API consists of routes and resources. Route direct incoming API requests to backend resources. A route consists of two parts: the HTTP method and the resource path, for example, GET /pets. You can define specific HTTP methods for routes. Alternatively, you can use ANY method to match all methods not yet defined for the resource. You can create a $default route to use as a “catch-all” method for requests that don’t match any other route.
The WebSocket API consists of routes and routing keys. Unlike REST APIs, which receive and respond to requests, WebSocket APIs support two-way communication between the client application and the backend. Backends can send callback messages to connected clients.
Private Integration: As described above, Integration (integration) is a key action in API Gateway. Private Integration emphasizes the integration of private resources placed in a VPC or resources in an internal network. Official document introduction:
Setting Up an API Gateway Private Integration – Amazon API Gateway
VPC Link: With VPC Link (VPC Link), you can create private integrations that connect HTTP API routes to private resources in a VPC, such as Network/Application Load Balancers or Amazon ECS container-based applications . Private integrations use VPC Links to encapsulate connections between API Gateway and target VPC resources. You can reuse VPC links across different routes and APIs. When creating a VPC Link, API Gateway creates and manages elastic network interfaces for the VPC Link in your account.
In fact, VPC Link is a technical component to realize Private Integration.
Private API: refers to the API exposed by API Gateway can only be accessed within the private network, not from the public network.
For the above Private Integration, Private API and VPC Link, here is another blog post for your reference: Understanding VPC links in Amazon API Gateway private integrations | AWS Compute Blog
Proxy Integration (proxy integration): With the all-encompassing proxy resource {proxy + } and REST/HTTP method, named ANY, it can match any REST/HTTP method, you can use the proxy Integrate to create an API with method ANY. This method exposes a complete set of accessible HTTP resources and operations on the backend. As the backend web server opens up more resources for public access, clients can access the new resources through this same API.
PrivateLink: Amazon PrivateLink is a highly available, scalable technology that enables private connection of your VPC to supported Amazon Web Services, services hosted by other Amazon accounts (via VPC endpoint node service) and support Amazon Cloud Technology Marketplace partner services. You do not need an internet gateway, NAT device, public IP address, Direct Connect connection, or Amazon Site-to-Site VPN connection to communicate with the service.
VPC Endpoint services: Your own application or service within your VPC.
VPC Endpoint: An entry point within a VPC that allows you to privately connect to a VPC Endpoint service.
3. Architecture diagram and problem definition
1) Architecture Diagram Introduction
1.1) Because in real scenarios, the API management platform will be designed in a separate account (Amazon Cloud Technology provides a multi-tenant environment), and because of the existence of objectively different business systems, they all need to be integrated with the API management platform , so the API management platform is likely to connect to different network environments, which may be other regions provided by Amazon Cloud Technology, offline environments, or other cloud environments. Therefore, the architecture diagram above designs a cross-account and cross-network environment.
1.2) Three accounts on the cloud are set here, namely API-Gateway Account, Application Account and Consumer Account. For the convenience of configuration, here Consumer and API-Gateway are in the same VPC in the same account. At the same time, there is also an Other Place to represent other regions, offline environments, or other cloud environments that may be provided by Amazon cloud technology.
2) Problem Definition
As shown in the architecture diagram above, the main issue discussed in this article is to use the API Gateway, through Private Integration, Private API, VPC Link, VPC Endpoint services, and VPC Endpoint. To complete cross-account or cross-network API integration in a private network environment.
Fourth, specific configuration
The configuration is mainly divided into two parts, one is to integrate with the cross-account internal application API, and the other is to integrate with internal network resources.
1. Integrate with cross-account internal application API
1) Configure Interface Endpoint Services (VPC Endpoint Services)
Log in to Application Account (01477********), select Load Balancers in EC2, and view the Service created through EKS (in the form of internal NLB), which is equivalent to purple 1 and 2 on the architecture diagram
Create a service powered by Amazon PrivateLink with reference to the official manual – Amazon Virtual Private Cloud
Complete the creation of the “Interface Endpoint Service”, which corresponds to purple 3 on the architecture diagram.
Select the service – Endpoint Services in the VPC, and click Create endpoint service
Select the Service created by EKS before (exposed by NLB)
Other options remain default.
Configuring Endpoint Services – Amazon Virtual Private Cloud
After you create an endpoint service configuration, you can control which service consumers can create interface endpoints to your service.
Enter the ARN of the IAM user or role in the Amazon account that is allowed to access the endpoint service of this interface. You can refer to the following official manual:
Configuring Endpoint Services – Amazon Virtual Private Cloud
The created “Interface Endpoint Service” is as follows, copy the Service name, which will be used later.
2) Configure the interface endpoint VPC Endpoint
Log in to the API-Gateway Account (84234********), select the service – Endpoint in the VPC, then click Create endpoint, enter the Name and the previously copied Service name, and Verify
service. This is equivalent to purple 4 and 5 on the architecture diagram.
Care should be taken when selecting VPC and Subnets here, as an Endpoint Network Interface (ENI) will be created in each of the selected subnets. An endpoint network interface is assigned a private IP address from within your subnet’s IP address range and maintains this IP address until the interface endpoint is deleted.
Note the created network interface (ENI) in the subnet, record the corresponding subnet and ip address.
3) Accept Endpoint connections
Log in to the Application Account (01477********), select Endpoint services in the VPC, and select the previously created Endpoint services. Select Accept endpoint connection request in Endpoint connections.
4) Create an NLB for the VPC link
Log in to the API-Gateway Account (84234********), select the service – Target groups in EC2, and then select IP addresses. The configuration is as follows, and the others remain default. This is equivalent to purple 6 on the architecture diagram.
In the next step, select the corresponding VPC for network, and enter the IP address recorded in the previous step for the IP address.
Log in to API-Gateway Account (84234********), select Services – Load balancers in EC2, and click Create at Network Load Balancer. Select Internal.
Select the corresponding VPC and subnet here.
After the creation is complete, record the NLB DNS name ( nlb-vpc-link-bbbd845d48779f09.elb.us-east-1.amazonaws.com )
5) Create a VPC Link
Log in to API-Gateway Account (84234********), select VPC links in Service – API Gateway, then click Create to select VPC link for REST APIs, and click Create. This is equivalent to purple 6 on the architecture diagram.
Select the NLB you just created in Target NLB:
Note that the status of the created VPC Link is Available, and record the VPC Link ID as nlb-vpc-link(wo74ch), which needs to be used next.
6) Prepare VPC Endpoint for API Gateway for creating Private Restful API
Let me explain here first, although they are all called VPC Endpoint, the VPC Endpoint here is to create an Interface VPC Endpoint for the Serverless service of API Gateway, that is, to create an ENI in the subnet in the specified VPC. The previously created VPC Endpoint is for accessing services exposed by other accounts through VPC Endpoint services. The common point is that they all use the technology of Private Link.
Log in to API-Gateway Account (84234********), select Service-Endpoints in VPC, and click Create endpoint. This is equivalent to purple 8 on the architecture diagram.
Record the created VPC Endpoint ID, which needs to be used next
7) Create a Private Restful API for cross-account resources
Log in to API-Gateway Account (84234********), select APIs in Service-API Gateway, then select REST API Private, and click Build. This is equivalent to purple 7 on the architecture diagram.
The configuration is as shown in the figure below, select Private at Endpoint Type, VPC Endpoint IDs and enter the VPC Endpoint ID recorded in the previous step.
Next, let’s create resources (Resource) and methods (Method)
Create a Resource, check Configure as proxy resource and Enable API Gateway CORS, because Configure as proxy resource is selected, so configure /{proxy + } as a proxy resource to capture all requests for its sub-resources. For example, it works for a GET request to /foo. To handle requests to /, add a new ANY method on the / resource. Otherwise, resource access can only be accessed through the path https://.execute-api..amazonaws.com/test/{proxy + }, not through the path https://.execute-api..amazonaws.com/test/ Access resources.
Configure private integration in the pop-up page, the specific configuration is as follows, select VPC Link in the Integration type, check Use Proxy Integration, select the VPC Link created before in the VPC Link – nlb-vpc-link(wo74ch).Endpoint URL input NLB DNS name( nlb-vpc-link-bbbd845d48779f09.elb.us-east-1.amazonaws.com ). Note that the Endpoint URL parameter here is not used to route the request to the endpoint, but to set the Host header and certificate verify.
Also create ANY method under/under the above steps. Otherwise, resource access can only be accessed through the path https://.execute-api..amazonaws.com/test/{proxy + }, not through the path https://.execute-api..amazonaws.com/test/ Access resources.
Before deploying the API, you need to create a Resource Policy to limit and protect what conditions your API can be accessed, such as restricting the ip address and VPC, which can only be accessed through the API Gateway VPC endpoint. Select Resource Policy and enter the following:
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Deny",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "execute-api:/*",
"Condition": {
"StringNotEquals": {
"aws:sourceVpce": "vpce-0416bbea8fb2bd63a"
}
}
},
{
"Effect": "Allow",
"Principal": "*",
"Action": "execute-api:Invoke",
"Resource": "execute-api:/*"
}
]
}
</code><em>Copy Code</em>
“aws:sourceVpce”: “vpce-0416bbea8fb2bd63a” restricts access to this API only from the VPC endpoint named vpce-0416bbea8fb2bd63a.
Deploy the API via Deploy API:
Due to the restriction of Resource Policy, it can only be accessed through the private network environment.
8) Test Private Restful API through private network across accounts
Now simulate api access from Consumer Account – https://3eqr0osoc3.execute-api.us-east-1.amazonaws.com/test
Test success.
2. Integration with internal network resources
Integration with internal network resources can be divided into two situations
(1) The resources that need to be integrated are accessed through IP addresses
(2) The resources that need to be integrated are accessed through Endpoint/URL
Next, we discuss that the resources to be integrated are accessed through IP addresses, and the specific configuration process is as follows:
1) Make sure the network is open
In the architecture diagram, a network connection has been established with offline or other network environments through Transit Gateway and Direct connect. Here I am simulating this scenario through the same Amazon cloud technology account across different regions.
This is equivalent to red 2, 3, 4, 5 on the architecture diagram.
I have already deployed the EC2 environment of nginx in Other Region to simulate the environment in Other Place
(Amazon other Region/On-premise/3rd location) resource with IP address 172.31.12.59. This is equivalent to red 1 on the architecture diagram.
2) Use the previously created VPC Endpoint for API Gateway
You can also refer to the previous steps:
6) Prepare VPC Endpoint for API Gateway for creating Private Restful API
This is equivalent to the red 9 on the architecture diagram.
3) Create an NLB prepared for VPC link, log in to API-Gateway Account (84234********), select Service – Target groups in EC2, then select IP addresses, and select Already connected to Other Place
(Amazon other Region/On-premise/3rd location) The connected VPC is established, and the protocol is TCP. The configuration is as follows, and the others remain default.
In the next step, select the corresponding Other private IP address for network, and enter the IP address in Other Place
IP address 172.31.12.59 of the resource EC deployed in (Amazon other Region/On-premise/3rd location). Then click Include as pending below. Then you can see the added Target in Review Targets.
Log in to API-Gateway Account(84234********), select Services – Load balancers in EC2, and click Create at Network Load Balancer. Select Internal.
Select the corresponding VPC and subnet here.
Select the Target Group you just created.
After the creation is complete, record the NLB DNS name ( nlb-other-network-137d7c7c7e159265.elb.us-east-1.amazonaws.com )
4) Create a VPC Link
Log in to API-Gateway Account (84234********), select VPC links in Service-API Gateway, then click Create to select VPC link for REST APIs, and click Create. This is equivalent to red 6,7 on the architecture diagram. Select the NLB you just created in Target NLB:
Note that the status of the created VPC Link is Available, and record the VPC Link ID as nlb-other-network (6h953x), which needs to be used next.
5) Create a Private Restful API for cross-network resources
Log in to API-Gateway Account (84234********), select APIs in Service-API Gateway, then select REST API Private, and click Build. This is equivalent to the red 8 on the architecture diagram.
The configuration is as shown in the figure below, select Private at Endpoint Type, VPC Endpoint IDs and enter the VPC Endpoint ID created before.
Next, let’s create resources (Resource) and methods (Method)
Create a Resource, check Configure as proxy resource and Enable API Gateway CORS, because Configure as proxy resource is selected, so configure /{proxy + } as a proxy resource to capture all requests for its sub-resources. For example, it works for a GET request to /foo. To handle requests to /, add a new ANY method on the / resource. Otherwise, resource access can only be accessed through the path https://.execute-api..amazonaws.com/test/{proxy + }, not through the path https://.execute-api..amazonaws.com/test/ Access resources.
Configure private integration in the pop-up page, the specific configuration is as follows, select VPC Link in the Integration type, check Use Proxy Integration, select the VPC Link created before in the VPC Link – nlb-other-network (6h953x).Endpoint URL Enter the previous step Prepared NLB DNS name ( nlb-other-network-137d7c7c7e159265.elb.us-east-1.amazonaws.com ). Note that the Endpoint URL parameter here is not for routing the request to an endpoint, but for setting the Host header and certificate validation.
Also create ANY method under/under the above steps. I won’t go into details here.
You need to create Resource Policy and Deploy API before Deploy API, just follow the previous steps.
Deploy the API via Deploy API:
Serverless
The created API is as follows:
5) Test the Private Restful API over the private network across the network
Now simulate the access from the Consumer Account to the api – https://buw9ocq5f5.execute-api.us-east-1.amazonaws.com/newtest and the test is successful.
5. Summary
Through the above steps, we have completed the integration of cross-accounts and internal application APIs through the Amazon cloud technology API management platform API Gateway, and also through integration with internal network resources. It satisfies the API management and integration through Rest in the case of private network.
Reference material:
Set up an API Gateway API with private integration using the AWS CLI – Amazon API Gateway
Amazon API Gateway – Getting Started with Amazon Web Services
Setting Up an API Gateway Private Integration – Amazon API Gateway
Understanding VPC links in Amazon API Gateway private integrations | AWS Compute Blog
Create a Service Powered by Amazon PrivateLink – Amazon Virtual Private Cloud
Configuring Endpoint Services – Amazon Virtual Private Cloud
The author of this article
Jin Zhongmin
Amazon Solutions Architect, currently working on cloud computing solutions and architectures. With more than 15 years of experience in the IT industry, he has been engaged in software development, after-sales support, system delivery, pre-sales, etc. Participated in many large-scale project architecture design and implementation delivery.
Article source: https://dev.amazoncloud.cn/column/article/630a3483d4155422a4610a62?sc_medium=regulartraffic &sc_campaign=crossplatform &sc_channel=CSDN
The knowledge points of the article match the official knowledge files, and you can further learn relevant knowledge Java skill treeHomepageOverview 118809 people are studying systematically