Kong Releases AI Gateway 3.8 with Semantic Intelligence for Faster, Smarter and More Secure GenAI Applications

News provided by

Sep 11, 2024, 09:15 ET

AI Gateway 3.8 Ships with Innovative Semantic Caching, Advanced Load Balancing and Semantic Prompt Guard For Faster AI Responses, Lower Costs and Enhanced Security

SAN FRANCISCO, Sept. 11, 2024 /PRNewswire/ -- Kong Inc., a leading developer of cloud API technologies, today announced the release of Kong AI Gateway 3.8, a groundbreaking upgrade that brings new semantic capabilities to the AI and API infrastructure space. This latest version, launched at API Summit 2024, is designed to make GenAI up to 20x faster, more efficient and more cost-effective while helping ensure robust security and governance.

"Kong AI Gateway 3.8 is a significant step forward in enabling organizations to fully harness the power of AI," said Marco Palladino, Co-Founder and Chief Technology Officer, Kong Inc. "By introducing Semantic Intelligence into our AI Gateway, we are addressing the most pressing challenges faced by enterprises leveraging GenAI today — speed, cost and security — while dramatically improving the developer experience."

Key Features and Capabilities of Kong AI Gateway 3.8:

The new Semantic Caching feature revolutionizes response times by recognizing when different user prompts essentially mean the same thing. For example, consider the questions "how long does it take to cook pasta?" and "how long does it take to cook spaghetti?" Instead of processing each prompt separately, the AI Gateway returns the same cached response, drastically cutting down processing time and reducing computational overhead. This enhancement makes Kong's AI Gateway faster and more efficient, enabling users to get the answers they need more quickly. In fact, with Semantic Caching enabled, responses can be up to 20x faster, delivering the best end-user AI experiences and making Kong's gateway a performance leader in its category.

Moreover, GenAI can be expensive, with each prompt consuming significant compute and energy resources. By reducing the number of redundant prompts at the gateway layer, Semantic Caching allows enterprises to potentially lower the cost of GenAI capabilities. In many instances, the AI Gateway can return cached content instead of making new requests for synonymous prompts. This can lead to cost savings, especially when managing large-scale AI deployments.

Kong AI Gateway 3.8 also introduces the Semantic Prompt Guard, a powerful new tool that brings intelligence to AI security. This capability allows the AI Gateway to understand the "essence" of user requests and block inappropriate or sensitive prompts, without needing to specify individual keywords. For instance, if an organization wants to block political content, the AI Gateway can prevent related terms from being processed, aiding compliance and reducing the risk of data breaches or misuse.

Additionally, with Kong AI Gateway 3.8, developers no longer need to navigate the complexities of selecting the best LLM for their tasks. The new Semantic Routing feature intelligently selects the most suitable LLM based on the developer's requirements, streamlining the development process and accelerating the adoption of GenAI within organizations. Support for Gemini and Bedrock LLMs further expands the AI Gateway's capabilities, enabling seamless integration with AWS and Google's AI ecosystems.

Here are some of the ways that Kong AI Gateway 3.8 enhances business value for enterprises looking to leverage GenAI:

Enhanced Customer Engagement and Time to Value: Semantic Caching improves the end-user experience by enabling AI applications to run up to 20x faster, resulting in more responsive and efficient interactions. This drives greater customer engagement and accelerates the time to value for organizations implementing AI.
Optimized Total Cost of Ownership: By leveraging Semantic Caching, organizations can significantly reduce the costs associated with GenAI, making AI deployments more sustainable and budget-friendly.
Developer Productivity and Experience: Advanced load balancing and Semantic Routing enhance developer productivity by simplifying the process of AI model selection and management, allowing teams to focus on creating business value rather than navigating the complexities of AI infrastructure.
Comprehensive Security and Governance: With the addition of Semantic Prompt Guard, Kong AI Gateway 3.8 provides unparalleled security and governance capabilities, helping ensure that AI applications remain compliant and secure in a rapidly evolving digital landscape.

AI Gateway 3.8 is also available with Kong Konnect, the API platform for AI, which is designed to help organizations securely build, run and govern AI-driven applications. With a focus on performance, security, governance and cost efficiency, the platform enables businesses to innovate faster by efficiently managing and scaling their API infrastructure.

For more information on AI Gateway 3.8, visit https://konghq.com/products/kong-ai-gateway and to learn more about Kong Konnect, please visit https://konghq.com/products/kong-konnect.

About Kong Inc.

Kong Inc., a leading developer of cloud API technologies, is on a mission to enable companies around the world to become "API-first" and securely accelerate AI adoption. Kong helps organizations globally — from startups to Fortune 500 enterprises — unleash developer productivity, build securely and accelerate time to market. For more information about Kong, please visit www.konghq.com or follow us on X @thekonginc.

SOURCE Kong Inc.