CloudTalk

Tag: Standardization

  • Agile AI Data Centers: Fungible Architectures for the AI Era

    Agile AI Data Centers: Fungible Architectures for the AI Era

    Agile AI Architectures: Building Fungible Data Centers for the AI Era

    Artificial Intelligence (AI) is rapidly transforming every aspect of our lives, from healthcare to software engineering. Innovations like Google’s Magic Cue on the Pixel 10, Nano Banana Gemini 2.5 Flash image generation, Code Assist, and Deepmind’s AlphaFold highlight the advancements made in just the past year. These breakthroughs are powered by equally impressive developments in computing infrastructure.

    The exponential growth in AI adoption presents significant challenges for data center design and management. At Google I/O, it was revealed that Gemini models process nearly a quadrillion tokens monthly, with AI accelerator consumption increasing 15-fold in the last 24 months. This explosive growth necessitates a new approach to data center architecture, emphasizing agility and fungibility to manage volatility and heterogeneity effectively.

    Addressing the Challenges of AI Growth

    Traditional data center planning involves long lead times that struggle to keep pace with the dynamic demands of AI. Each new generation of AI hardware, such as TPUs and GPUs, introduces unique power, cooling, and networking requirements. This rapid evolution increases the complexity of designing, deploying, and maintaining data centers. Furthermore, the need to support various data center facilities, from hyperscale environments to colocation providers across multiple regions, adds another layer of complexity.

    To address these challenges, Google, in collaboration with the Open Compute Project (OCP), advocates for designing data centers with fungibility and agility as core principles. Modular architectures, interoperable components, and the ability to late-bind facilities and systems are essential. Standard interfaces across all data center components—power delivery, cooling, compute, storage, and networking—are also crucial.

    Power and Cooling Innovations

    Achieving agility in power management requires standardizing power delivery and building a resilient ecosystem with common interfaces at the rack level. The Open Compute Project (OCP) is developing technologies like +/-400Vdc designs and disaggregated solutions using side-car power. Emerging technologies such as low-voltage DC power and solid-state transformers promise fully integrated data center solutions in the future.

    Data centers are also being reimagined as potential suppliers to the grid, utilizing battery-operated storage and microgrids. These solutions help manage the “spikiness” of AI training workloads and improve power efficiency. Cooling solutions are also evolving, with Google contributing Project Deschutes, a state-of-the-art liquid cooling solution, to the OCP community. Companies like Boyd, CoolerMaster, Delta, Envicool, Nidec, nVent, and Vertiv are showcasing liquid cooling demos, highlighting the industry’s enthusiasm.

    Standardization and Open Standards

    Integrating compute, networking, and storage in the server hall requires standardization of physical attributes like rack height, width, and weight, as well as aisle layouts and network interfaces. Standards for telemetry and mechatronics are also necessary for building and maintaining future data centers. The Open Compute Project (OCP) is standardizing telemetry integration for third-party data centers, establishing best practices, and developing common naming conventions and security protocols.

    Beyond physical infrastructure, collaborations are focusing on open standards for scalable and secure systems:

    • Resilience: Expanding manageability, reliability, and serviceability efforts from GPUs to include CPU firmware updates.
    • Security: Caliptra 2.0, an open-source hardware root of trust, defends against threats with post-quantum cryptography, while OCP S.A.F.E. streamlines security audits.
    • Storage: OCP L.O.C.K. provides an open-source key management solution for storage devices, building on Caliptra’s foundation.
    • Networking: Congestion Signaling (CSIG) has been standardized, improving load balancing. Advancements in SONiC and efforts to standardize Optical Circuit Switching are also underway.

    Sustainability Initiatives

    Sustainability is a key focus. Google has developed a methodology for measuring the environmental impact of AI workloads, demonstrating that a typical Gemini Apps text prompt consumes minimal water and energy. This data-driven approach informs collaborations within the Open Compute Project (OCP) on embodied carbon disclosure, green concrete, clean backup power, and reduced manufacturing emissions.

    Community-Driven Innovation

    Google emphasizes the power of community collaborations and invites participation in the new OCP Open Data Center for AI Strategic Initiative. This initiative focuses on common standards and optimizations for agile and fungible data centers.

    Looking ahead, leveraging AI to optimize data center design and operations is crucial. Deepmind’s AlphaChip, which uses AI to accelerate chip design, exemplifies this approach. AI-enhanced optimizations across hardware, firmware, software, and testing will drive the next wave of improvements in data center performance, agility, reliability, and sustainability.

    The future of data centers in the AI era depends on community-driven innovation and the adoption of agile, fungible architectures. By standardizing interfaces, promoting open collaboration, and prioritizing sustainability, the industry can meet the growing demands of AI while minimizing environmental impact. These efforts will unlock new possibilities and drive further advancements in AI and computing infrastructure.

    Source: Cloud Blog

  • Agile AI: Google’s Fungible Data Centers for the AI Era

    Agile AI: Google’s Fungible Data Centers for the AI Era

    Agile AI Architectures: A Fungible Data Center for the Intelligent Era

    Artificial intelligence (AI) is rapidly transforming every aspect of our lives, from healthcare to software engineering. Google has been at the forefront of these advancements, showcasing developments like Magic Cue on the Pixel 10, Nano Banana Gemini 2.5 Flash image generation, Code Assist, and AlphaFold. These breakthroughs are powered by equally impressive advancements in computing infrastructure. However, the increasing demands of AI services require a new approach to data center design.

    The Challenge of Dynamic Growth and Heterogeneity

    The growth in AI is staggering. Google reported a nearly 50X annual growth in monthly tokens processed by Gemini models, reaching 480 trillion tokens per month, and has since seen an additional 2X growth, hitting nearly a quadrillion monthly tokens. AI accelerator consumption has grown 15X in the last 24 months, and Hyperdisk ML data has grown 37X since GA. Moreover, there are more than 5 billion AI-powered retail search queries per month. This rapid growth presents significant challenges for data center planning and system design.

    Traditional data center planning involves long lead times, but AI demand projections are now changing dynamically and dramatically, creating a mismatch between supply and demand. Furthermore, each generation of AI hardware, such as TPUs and GPUs, introduces new features, functionalities, and requirements for power, rack space, networking, and cooling. The increasing rate of introduction of these new generations complicates the creation of a coherent end-to-end system. Changes in form factors, board densities, networking topologies, power architectures, and liquid cooling solutions further compound heterogeneity, increasing the complexity of designing, deploying, and maintaining systems and data centers. This also includes designing for a spectrum of data center facilities, from hyperscale to colocation providers, across multiple geographical regions.

    The Solution: Agility and Fungibility

    To address these challenges, Google proposes designing data centers with fungibility and agility as primary considerations. Architectures need to be modular, allowing components to be designed and deployed independently and be interoperable across different vendors or generations. They should support the ability to late-bind the facility and systems to handle dynamically changing requirements. Data centers should be built on agreed-upon standard interfaces, so investments can be reused across multiple customer segments. These principles need to be applied holistically across all components of the data center, including power delivery, cooling, server hall design, compute, storage, and networking.

    Power Management

    To achieve agility and fungibility in power, Google emphasizes standardizing power delivery and management to build a resilient end-to-end power ecosystem, including common interfaces at the rack power level. Collaborating with the Open Compute Project (OCP), Google introduced new technologies around +/-400Vdc designs and an approach for transitioning from monolithic to disaggregated solutions using side-car power (Mt. Diablo). Promising technologies like low-voltage DC power combined with solid state transformers will enable these systems to transition to future fully integrated data center solutions.

    Google is also evaluating solutions for data centers to become suppliers to the grid, not just consumers, with corresponding standardization around battery-operated storage and microgrids. These solutions are already used to manage the “spikiness” of AI training workloads and for additional savings around power efficiency and grid power usage.

    Data Center Cooling

    Data center cooling is also being reimagined for the AI era. Google announced Project Deschutes, a state-of-the-art liquid cooling solution contributed to the Open Compute community. Liquid cooling suppliers like Boyd, CoolerMaster, Delta, Envicool, Nidec, nVent, and Vertiv are showcasing demos at major events. Further collaboration is needed on industry-standard cooling interfaces, new components like rear-door-heat exchangers, and reliability. Standardizing layouts and fit-out scopes across colocation facilities and third-party data centers is particularly important to enable more fungibility.

    Server Hall Design

    Bringing together compute, networking, and storage in the server hall requires standardization of physical attributes such as rack height, width, depth, weight, aisle widths, layouts, rack and network interfaces, and standards for telemetry and mechatronics. Google and its OCP partners are standardizing telemetry integration for third-party data centers, including establishing best practices, developing common naming and implementations, and creating standard security protocols.

    Open Standards for Scalable and Secure Systems

    Beyond physical infrastructure, Google is collaborating with partners to deliver open standards for more scalable and secure systems. Key highlights include:

    • Resilience: Expanding efforts on manageability, reliability, and serviceability from GPUs to include CPU firmware updates and debuggability.
    • Security: Caliptra 2.0, the open-source hardware root of trust, now defends against future threats with post-quantum cryptography, while OCP S.A.F.E. makes security audits routine and cost-effective.
    • Storage: OCP L.O.C.K. builds on Caliptra’s foundation to provide a robust, open-source key management solution for any storage device.
    • Networking: Congestion Signaling (CSIG) has been standardized and is delivering measured improvements in load balancing. Alongside continued advancements in SONiC, a new effort is underway to standardize Optical Circuit Switching.

    Sustainability

    Sustainability is embedded in Google’s work. They developed a new methodology for measuring the energy, emissions, and water impact of emerging AI workloads. This data-driven approach is applied to other collaborations across the OCP community, focusing on an embodied carbon disclosure specification, green concrete, clean backup power, and reduced manufacturing emissions.

    AI-for-AI

    Looking ahead, Google plans to leverage AI advances in its own work to amplify productivity and innovation. Deepmind AlphaChip, which uses AI to accelerate and optimize chip design, is an early example. Google sees more promising uses of AI for systems across hardware, firmware, software, and testing; for performance, agility, reliability, and sustainability; and across design, deployment, maintenance, and security. These AI-enhanced optimizations and workflows will bring the next order-of-magnitude improvements to the data center.

    Conclusion

    Google’s vision for agile and fungible data centers is crucial for meeting the dynamic demands of AI. By focusing on modular architectures, standardized interfaces, power management, liquid cooling, and open compute standards, Google aims to create data centers that can adapt to rapid changes and support the next wave of AI innovation. Collaboration within the OCP community is essential to driving these advancements forward.

    Source: Cloud Blog