Glossary and References

Below is a list of terms and concepts that could be defined for common usage. Feel free to add to the list of terms, and to begin offering possible definitions.

Action

Operation that is executed using resources, ordered in a way that maintains dependences, has a specific beginning and end (we need to make this more rigorous) Actions may be a different kinds.

• Compute (task)
• Data movement
• Control (synchronization, conditional/predication)
• Resources management (compute, memory, …), e.g. allocation, physical materialization, deallocation/freeing

Event

Handle for an object that tracks whether sync has been triggered, that may be used for communicating the fulfillment of data or control dependences

Hierarchical

• A hierarchy may have some number of levels, some number of actions at each level
• A hierarchical design facilitates commonality of abstraction across different layers
• Kinds of hierarchy
  • Data - partitioning of data can imply decomposition of work
  • Control - an aggregation operation can be broken into pieces, and those pieces could range from a simple sequence to a complex task graph with cycles with control
  • Resource - based on topological groups, for example

Heterogeneous

Heterogeneity comes in different forms

• Kinds of resources
  • Compute, e.g. CPU, GPU, FPGA, TPU, ASIC
  • Memory, e.g. HBM, DDR, SW-managed cache, level in caching hierarchy
• Numbers of resources
  • Computing elements
  • Memory bytes
• Arrangements of resources
  • Topology of computing elements
  • Data layout
• Implementations
  • Network or connectivity configurations, e.g. choice of fabric
  • Algorithm
  • Code implementation, e.g. for different data layout, different computing kid, number of computing elements, amount of memory, etc.