PRR Project

Summary

Artificial Intelligence and Big Data Analytics are re-shaping the future of our society. Indeed, extracting useful and insightful information from data is key for the competitiveness and efficiency of critical sectors such as Health, Finance, e-Governance, Science. However, as the amount of data generated worldwide grows, so does the complexity of analyzing it, requiring specialized hardware, hundreds of servers, and a large capacity to store the information being processed and the outputs being generated. A good example is the current trend on Large Language Models (LLMs), similar to ChatGPT, that require Petabytes of data, and computing power from hundreds to thousands of servers, to be effective.This is where High-Performance Computing  (HPC) centers play a crucial role, i.e., by providing high-end hardware (GPUs, TPUs), and large-scale resources (e.g., in the order of tens of thousands of computing nodes, and several petabytes of storage capacity). With such resources, these hyperscale infrastructures can accommodate demanding workloads from multiple users simultaneously. However, as infrastructures get larger, and applications more demanding, one faces two main challenges.Finding an optimal approach for managing available computing and storage resources becomes increasingly difficult. There are several workloads, from different users, running at the same time and competing for HPC resources. This can easily lead to performance bottleneck scenarios where the whole HPC cluster becomes unavailable. For instance, this is happening in large supercomputers from TACC (USA) and AIST (Japan), where bursts of data access lead to severe contention at shared storage resources and, eventually, to service downtime.Large clusters, and demanding applications ( e.g., requiring GPUs) are known for consuming significant energy. Although performance is a key concern of HPC centers, one cannot forget the need to lower our carbon footprint and have more sustainable solutions. Therefore, the management of computing and storage resources cannot be made only from the point of view of performance and scale, it also needs to have energy efficiency in mind.These two research challenges justify the proposed job, which will focus on advancing science and technology in two main focal research questions: i) how can one make hyperscale infrastructure more efficient, in terms of performance and scale, when handling today´s demanding workloads that require high computational power and large storage capacity?; ii) how can one achieve performance and scalability, while still guaranteeing sustainability and energy efficiency for these large-scale infrastructures?Our research on development in this area is directly contributing to Sustainable Development Goals of the United Nations 2030 Agenda 7, 9, and 12 in the context of projects such as: Sustainable HPC - Highly Sustainable Performance Computing (FAI/FEE), Epicure (DIGITAL-EUROHPC-JU-2022-APPSUPPORT-01) and HANAMI (EU-Japan Partnership). Informatics Department and INESC TEC have hosted, in the scope of these projects, a number of young researchers / teaching assistants that are key to their success and would be eligible for an FCT Tenure position.In this context, the required profile is as follows:A Ph.D. degree in Computer Science, advanced computing systems or related field, obtained less than 10 years ago;Demonstrated the ability to make original contributions to the state of the art in large-scale/hyperscale systems and infrastructures, in particular, on computational and storage resources management and efficiency.Having experience in leading or participating in research teams, in particular, in the context of collaborative research projects and the application project funding entities;Experience in teaching and supervising students in topics related to distributed infrastructures ( e.g., Cloud Computing, HPC) and large-scale systems;Experience in development and technology transfer activities towards achieving wider economic and societal impact.It is expected that the prospective employee is assigned the following tasks:contribution to research on aspects of computational and storage resources management and efficiency within the scope of distributed infrastructures ( e.g., Cloud Computing, HPC) and large-scale systems fundamentals, techniques, and tools;promotion of novel research and development opportunities that advance the state of the art and realize its impact in industry and society;supervision of junior researchers on topics related to computational and storage resources management and efficiency;development of topics related to computational and storage resources management and efficiency in the teaching curricula of distributed infrastructures ( e.g., Cloud Computing, HPC) and large-scale systems  courses at bachelor and master levels.