Glossary and References

Below is a list of terms and concepts that could be defined for common usage. Feel free to add to the list of terms, and to begin offering possible definitions.

Action

Operation that is executed using resources, ordered in a way that maintains dependences, has a specific beginning and end (we need to make this more rigorous) Actions may be a different kinds.

• Compute (task)
• Data movement
• Control (synchronization, conditional/predication)
• Resources management (compute, memory, …), e.g. allocation, physical materialization, deallocation/freeing

Data source, origin

Originator of data communication; where data is transferred from. The set of resources with which the source is associated is called the origin, e.g. the origin process, according to the MPI spec.

Data sink

Destination for data communication

Event

Handle for an object that tracks whether sync has been triggered, that may be used for communicating the fulfillment of data or control dependences

Task

Unit of execution that may be split into one or many tasklets.

Tasklet

Non-decomposable unit of execution that performs useful work.

Hierarchical

• A hierarchy may have some number of levels, some number of actions at each level
• A hierarchical design facilitates commonality of abstraction across different layers
• Kinds of hierarchy
  • Data - partitioning of data can imply decomposition of work
  • Control - an aggregation operation can be broken into pieces, and those pieces could range from a simple sequence to a complex task graph with cycles with control
  • Resource - based on topological groups, for example

Heterogeneous

Heterogeneity comes in different forms

• Kinds of resources
  • Compute, e.g. CPU, GPU, FPGA, TPU, ASIC
  • Memory, e.g. HBM, DDR, SW-managed cache, level in caching hierarchy
• Numbers of resources
  • Computing elements
  • Memory bytes
• Arrangements of resources
  • Topology of computing elements
  • Data layout
• Implementations
  • Network or connectivity configurations, e.g. choice of fabric
  • Algorithm
  • Code implementation, e.g. for different data layout, different computing kid, number of computing elements, amount of memory, etc.

Directory

Look up service to identify a sink or source in a lower-level communication substrate.

It might be wise to externalize a global directory service and make it an optional part of the reference implementation. By so doing, we allow a cottage industry to create competing implementation of such look-up services that serve specific needs of applications, hardware, or the middleware.