Tim Mattson - Four selected lectures
Hinweis: Die Veranstaltung wird vollständig auf Englisch stattfinden.
The lectures
OpenMP: Its more than “parallel for” on SMP systems
Those of us close to OpenMP don’t like it, but for the majority of programmers, OpenMP is an easy way to get some extra performance on a multicore system without much effort. We will back this depressing conclusion with data from the HPCorpus dataset. Most OpenMP programmers treat systems as Symmetric Multiprocessors (SMP) and limit parallelization to compute-intensive loops with a parallel for directive (with reduction and a few other simple clauses).
In this lecture, we will discuss OpenMP for a wider range of systems with a focus of non-uniform memory architectures (NUMA) and GPU programming. Time permiSng, we will also dive into the details of the OpenMP memory model and how it impacts point-to-point synchronization.
Parallel Programming with Python
Old timers (like me) are not happy about this, but the most popular language used today is Python. Usage of the C, C++ and Fortran languages HPC people “love” is almost insignificant compared to the masses of programmers using Python. Of course, that is those python programmers’ problem. Real programmers use a low-level language (such as C). They just have to adapt and learn a “grown-up” language.
That perspective, of course, is ridiculous. For those of us working on parallel programming models, it is never our job to tell people which language to use. It is our job to figure out how to make people productive using the languages they choose.
He is also a recently retired kayak coach and instructor trainer (ACA certified). His obsession with sea kayaking, including “self-wetting” moments in the ocean, is pretty bad.
So how do we make Python fast and useful for parallel programming? That is the topic of this lecture. I’ll focus on my own research to incorporate OpenMP into Python. But then we’ll discuss the other commonly used options in use today and a bit about the future of python for parallel programming.
Floating Point Arithmetic is not real and your random numbers aren’t random
This lecture should be required for anyone who does arithmetic on a computer or builds algorithms around random sampling. Long ago when computational science was young, those of us working in the field learned numerical analysis. If we wanted an algorithm based on random sampling, we wrote our own random number generators. In other words, we were forced to take the time to understand how numbers work on computers.
Now days, these topics are rarely taught. In many ways, this is a good thing. There is so much to learn before you can do computational science, who has time to worry about numbers? That aStude works fine until it doesn’t. We will talk about how numbers can fail you and how to avoid contributing to disaster or the embarrassment of publishing bad research results. There isn’t much you must know. We’ll cover it all in this lecture.
The Future and what to do about it: People, processors, and programming
Hardware trends are clear. Driven by economics and the need to deliver increasing performance within a fixed power budget, computer systems are becoming increasingly complex. This complexity is directly exposed to software, hence, the need for programmers with a detailed understanding of computer architecture.
Unfortunately, programmers today are trained with programming languages that hide the hardware. You can’t specialize an algorithm to hardware features if there is a virtual machine between your code and the system or if you program in an interpreted language (such as python).
How are we going bridge this disconnect between our processors, the people who write our software, and the programming languages they use? We must fundamentally change how we construct software. We must automate key steps in software development using machine learning and AI technologies to map code onto the details of different systems.
In this talk, after describing the fundamentals of hardware system evolution, we’ll describe our work to automate key aspects of software development. We will describe successes and reasons for hope, but also fundamental challenges that limit the applicability of AI to address this problem.
The lecturer
Tim Mattson is a parallel programmer obsessed with every variety of science (Ph.D. Chemistry, UCSC, 1985). In 2023 he retired after a 45-year career in HPC (30 of which were with Intel). During his long career he has had the privilege of working with smart people on great projects including: (1) the first TFLOP computer (ASCI Red), (2) Parallel programming languages … Linda, Strand, MPI, OpenMP, OpenCL, OCR and PyOMP (3) two different research processors (Intel’s TFLOP chip and the 48 core SCC), (4) Data management systems (Polystore systems and the TileDB Array-based storage engine), (5) the GraphBLAS API for expressing graph algorithms as sparse linear algebra. Tim has over 150 publications including six books on different aspects of parallel computing.
He is also a recently retired kayak coach and instructor trainer (ACA certified). His obsession with sea kayaking, including “self-wetting” moments in the ocean, is pretty bad.