Summary Bullets:
• While both generative AI (GenAI) is having widespread interests, and more use cases are emerging, companies need to ensure the infrastructure does not become a bottleneck.
• GenAI will need new approaches from the processors to data centers and cooling systems to ensure the development can be efficient and sustainable.
AI continues to be a hot topic moving into 2024. The technology has evolved from predictive AI to GenAI and multi-modal GenAI, over a relatively short period of time. This technology is creating endless possibilities, and various use cases have started to emerge. For example, GenAI/AI have been applied for disease diagnosis, application development (assisted code development), marketing content creation, and demand forecasting for retail businesses. Against this backdrop, it is no surprise to see many forecasters expecting a rapid growth of revenue related to GenAI. However, amid the excitement, there have been concerns, particularly the need to address ethical issues and responsible AI. One area that AI technology providers have not been highlighting is the infrastructure, including the physical facility required to host the hardware for AI workloads.
The processing power required for GenAI workloads has pushed the semiconductor industry to develop faster and more efficient AI-optimized solutions. Training a large language model (LLM), for example, with ChatCPT can take a very long time with multiple GPUs (years with a single GPU). The industry is working on solutions such as Google’s TPU architecture and Nvidia’s Infiniband for faster networking speeds, interconnecting GPU servers. But besides achieving the processing speed, the hardware needs to be compact and consume less power. Most of the LLM development and training of AI models are being done in data centers, and this is driving demand for data center space as well as how it is being designed.
Yet, building new facilities in key data center hubs will get more difficult due to power requirements. The growing demand for energy will be a major concern as countries have commitments to meet carbon emission targets. For example, Singapore had a four-year moratorium that began in 2019 to stop the development of new data centers due to the high energy consumption, which will impede Singapore’s efforts to achieve its sustainability goals. While the country lifted the moratorium in mid-2023, it has been more selective in approving new data centers based on criteria such as energy efficiency, AI/ML compute and HPC capabilities, expansion of international connectivity, and the economic commitments to Singapore. In London (England), it was also reported that in some boroughs west of the city, the lack of sufficient ‘electricacapacity’ – due to data center demand – has impacted approvals for new housing developments. Power supply and sustainability considerations are also driving the data center industry to build more sustainable facilities through self-generation using fuel cells, renewables, and batteries. However, this will also require new power management strategy to switch from different energy sources as well as to cope with the surge in power demand when running GenAI workloads at scale.
Data centers packed with high-density servers will also require new systems to remove the heat. There are limitations with existing systems that are mostly based on air cooling. According to Vertiv, legacy facilities are ill-equipped to support widespread implementation of the high-density computing required for AI, with many lacking the required infrastructure for liquid cooling. Companies will need to retrofit their data centers, which requires additional investment. However, this is also an opportunity for companies to become more sustainable by adopting a more efficient cooling system. The International Energy Agency (IEA) estimates that data centers consume about 1-1.3% of the global final electricity demand, and this is expected to increase over time due to accelerating cloud adoption and AI development. While enterprises and policy makers are keen to harness the power of GenAI, they need to review data center requirements to pave the way for innovation happening at speed and scale.