HHAT Usage Meeting Minutes - Revision history

imported>Cnewburn at 16:54, July 18, 2017

2017-07-18T16:54:01Z

← Older revision		Revision as of 16:54, July 18, 2017
Line 3:		Line 3:

	A link to get back up to the parent page is [[Hierarchical_Heterogeneous_Asynchronous_Tasking \| here]].		A link to get back up to the parent page is [[Hierarchical_Heterogeneous_Asynchronous_Tasking \| here]].

			= BoF, Active Messages, POC Update, July 18, 2017 =
			Attendees (41):
			Wilf, Bala Seshasayee (Intel), Kath Knobe, Marcin Zalewski, Max Grossman, Rob Vander Wijngaart, Wael Elwasif (ORNL), Andrew Lumsdaine, BillF, Hartmut Kaiser, Jesun (PNNL), John Feo (PNNL), John Stone, Michael Garland, Michael Wong, Millad Ghane, minu455, Oscar Hernandez, Patrick Atkinson, Ron Brightwell, Siegfried Benkner (U Vienna), Sunita Chandrasekaran, Szilard Pall, Vincent Cave, Ashwin Aji (AMD), John Biddiscombe (CSCS), Naoya Maruyama, Kamil Halbiniak, Roman Wyrzykowski, CJ Newburn, Mike Bauer

			*BoF - BillF
			**SC17 BoFs due 2 weeks from today
			**60min mid day or 90min evening

			*Active messages - Marcin Zalewski
			**Overview, benefits
			**Active Pebbles/AM++ and GMT currently on MPI, tiny messages, latency/throughput trade-offs
			**Could be layered on PGAS - put with completion
			**Would like to look into integration with HiHAT
			*** Layering above HiHAT, mapping to transports under HiHAT
			*** Triggering upon receipt of transferred data
			*** Consider whether there are ordering requirements, e.g. fence/quiet
			**RonB: biggest differentiator of active messages is on receiver side
			**HartmutK: integration with scheduler is interesting part

			*Proof of Concept Update - Millad Ghane (U Houston), John Stone (UIUC)
			** Preliminary implementation to demonstrate feasibility
			** Implemented basics: data movement, data management, invocation
			** Current target platform: 2 CPU sockets, 2 GPUs
			** HiHAT overheads relative to CUDA for microbenchmarks: lesser of 2% or 500ns
			** User/Common Layer overhead for table look ups: 60ns
			** Molecular Orbitals app running on top
			*** 3% HiHAT API overhead
			*** 1.2x reduction in static APIs and unique APIs
			*** Porting time to HiHAT was 90 minutes


	= More users, proof of concept plans, high-level design doc, June 20, 2017 =		= More users, proof of concept plans, high-level design doc, June 20, 2017 =

imported>Cnewburn: /* More users, proof of concept plans, high-level design doc, June 20, 2017 */

2017-06-20T16:58:10Z

More users, proof of concept plans, high-level design doc, June 20, 2017

← Older revision		Revision as of 16:58, June 20, 2017
Line 6:		Line 6:
	= More users, proof of concept plans, high-level design doc, June 20, 2017 =		= More users, proof of concept plans, high-level design doc, June 20, 2017 =
	Attendees (43):		Attendees (43):
	Antonio Tumeo, Ashwin Aji, BillF, David Bernholdt, DebalinaB, DGenet, Dmitry Liakh, Firo017, Gordon Brown, Hans Johansen, James Beyer, Jesun (PNNL), Jiri Dokulil, John Stone, Kamil Halniniak and Roman, Kath Knobe, Keeran Brabazon (~~Leeds~~), Mauro Bianco, Michael Garland, Mike Bauer, Mike Chu, Millad Ghane, Minu455, Naoya Maruyama, Piotr, Rob Neely, Ronak Buch, Ruyman Reyes, Samuel Thibault, SharanA, Siegfried Benkner (U Vienna), Stephen Olivier, Szilard Pall, Thomas Herault, Tim Blattner, Vincent Cave, Wael Elwasif (ORNL), CJ, ...		Antonio Tumeo, Ashwin Aji, BillF, David Bernholdt, DebalinaB, DGenet, Dmitry Liakh, Firo017, Gordon Brown, Hans Johansen, James Beyer, Jesun (PNNL), Jiri Dokulil, John Stone, Kamil Halniniak and Roman, Kath Knobe, Keeran Brabazon (ARM), Mauro Bianco, Michael Garland, Mike Bauer, Mike Chu, Millad Ghane, Minu455, Naoya Maruyama, Piotr, Rob Neely, Ronak Buch, Ruyman Reyes, Samuel Thibault, SharanA (NVIDIA Tegra), Siegfried Benkner (U Vienna/StarPU), Stephen Olivier, Szilard Pall, Thomas Herault, Tim Blattner, Vincent Cave, Wael Elwasif (ORNL), CJ, ...

	* Some new participants: NIST/HTGS, UINTAH, StarPU, more from NVIDIA, e.g. automotive		* Some new participants: NIST/HTGS, UINTAH, StarPU, more from NVIDIA, e.g. automotive

imported>Cnewburn at 16:47, June 20, 2017

2017-06-20T16:47:58Z

@@ Line 3: / Line 3: @@
 A link to get back up to the parent page is [[Hierarchical_Heterogeneous_Asynchronous_Tasking | here]].
 = Mini-Summit Synthesis, May 16, 2017 =
 Attendees

imported>Cnewburn: /* Mini-Summit Synthesis, May 16, 2017 */

2017-05-16T16:54:17Z

Mini-Summit Synthesis, May 16, 2017

← Older revision		Revision as of 16:54, May 16, 2017
Line 5:		Line 5:
	= Mini-Summit Synthesis, May 16, 2017 =		= Mini-Summit Synthesis, May 16, 2017 =
	Attendees		Attendees
			Carter, David Bernholdt, George, Max, Michael Garland, Michael Robson (PPL/UIUC); Millad, Patrick, Piotr, Thomas Herault, Dmitry, Szilard, Toby, Wael, Damien, Andrew, Ashwin, Jiri Dokulil, Naoya Maruyama, Oscar, Pietro Cicotti, Wilf, CJ

	* Welcome, intro		* Welcome, intro
	* DHPC++ review		* DHPC++ review
	** Compare/contrast with OpenCL		** Compare/contrast with OpenCL, OpenVX, Vulcan
	* Mini-Summit review		* Mini-Summit review
	** Who gathered		** Who gathered
Line 15:		Line 16:
	** Tabulation of results		** Tabulation of results
	** Review of poll/ratification		** Review of poll/ratification
			*** This broader audience also ratified what was listed
			*** How do you connect different MPI worlds?
			*** Clarify that HiHAT has to stage data across sub-clusters
			*** Clarify granularity of work
	** Sampling of requirements		** Sampling of requirements
	*** Active messages (PNNL)		*** Active messages (PNNL, Andrew)
	*** Futures with data (OCR)		*** Futures with data (OCR, Vincent; HPX, Hartmut Kaiser)
	*** Callbacks on completion (OCR)		*** Callbacks on completion (OCR, Vincent)
	*** Dynamic compilation (R-Stream)		*** Dynamic compilation (R-Stream, KART, LLVM)
	*** Graph reuse (SWIFT/QuickShed)		*** Graph reuse (SWIFT/QuickShed, Stephen) - later
	*** Partial I/O (SWIFT/QuickShed)		*** Partial I/O (SWIFT/QuickShed)
	*** Feedback for auto-tuning (TensorRT)		*** Feedback for auto-tuning (TensorRT)
			*** Reproducibility via control
	** Additional key issues to debate		** Additional key issues to debate
	* Who else should be drawn in		* Who else should be drawn in
	** OpenVX		** OpenVX
	** Vulcan		** Vulcan
	** ~~Other?~~		** StarPU
	* ~~More detail~~		** UINTAH
	**		* Topics for the future
			** Portability, content of tasks - Carter
			** Task scheduling for accelerators, SMP - Szilard
			** Interoperation, remerging with other efforts, e.g. OpenCL, OpenMP - Szilard
			** Performance analysis and monitoring APIs - Oscar
			** Defining terms, e.g. future

	= Partner interactions, NVIDIA usage models, AMD efforts, Apr 18, 2017 =		= Partner interactions, NVIDIA usage models, AMD efforts, Apr 18, 2017 =

imported>Cnewburn at 15:51, May 16, 2017

2017-05-16T15:51:41Z

@@ Line 3: / Line 3: @@
 A link to get back up to the parent page is [[Hierarchical_Heterogeneous_Asynchronous_Tasking | here]].
 = Partner interactions, NVIDIA usage models, AMD efforts, Apr 18, 2017 =
 Attendees (37) included: Wilf, Kath Knobe, Millad Ghane, CJ Newburn, Bill Feiereisen, Dmitry Liakh, Gordon Brown, Jesmin Tithi, Jans Johansen, Jiri Dokulil, John Feo, Kelly Livingston, Max grossman, Mauro Bianco, Oscar Hernandez, Patrick Atkinson, Piotr Luszczek, Ron Brightwell, Ruyman Reyes, Stephen Jones, Stephen Olivier, Sunita Chandrasekaran, Szilard Pall, Wael Elwasif, Ashwin Aji, George Bosilca, John Stone, Benoit Meister, Andrew Lumsdaine, Mike Bauer, Tim

imported>Cnewburn: /* Partner interactions, NVIDIA usage models, AMD efforts, Apr 18, 2017 */

2017-04-18T16:59:02Z

Partner interactions, NVIDIA usage models, AMD efforts, Apr 18, 2017

← Older revision		Revision as of 16:59, April 18, 2017
Line 12:		Line 12:
	** David Keyes, on categories of hierarchical algorithms		** David Keyes, on categories of hierarchical algorithms
	* DHPC++ workshop, Toronto, May 16; will be a talk on HiHAT https://easychair.org/cfp/dhpcc17		* DHPC++ workshop, Toronto, May 16; will be a talk on HiHAT https://easychair.org/cfp/dhpcc17
			* Performance portability workshop, week of Aug 21, is expected to have some coverage of HiHAT

	Upcoming HiHAT Mini-Summit		Upcoming HiHAT Mini-Summit
Line 24:		Line 25:
	** reduce overheads for actions like invocation and moving data, instigated by CPU and performed on GPU --> lower-overhead Common Layer		** reduce overheads for actions like invocation and moving data, instigated by CPU and performed on GPU --> lower-overhead Common Layer
	** aggregate tasks in sequences and sub-graphs, that are passed down to target for localized handling --> richer tasking abstractions		** aggregate tasks in sequences and sub-graphs, that are passed down to target for localized handling --> richer tasking abstractions
			* Common requirements induced by inference and deep learning for HiHAT

	Teaser on relevant AMD efforts, Ashwin Aji, AMD		Teaser on relevant AMD efforts, Ashwin Aji, AMD
	* ROCm and ATMI		* ROCm = Radeon Open Compute, rebranding of HSA
			** Similar to common layer, thin API that abstracts underlying compute and memory HW
			** Task descriptors, lock-free data structures, door bells that trigger task execution
			* ATMI = Async Tasking and Memory Interface
			** Kinds of tasks, low-latency signaling among tasks
			* Links
			** ATMI info: http://gpuopen.com/compute-product/atmi/
			** ATMI github: https://github.com/RadeonOpenCompute/atmi
			** ROCm platform info: http://gpuopen.com/compute-product/rocm/
			** ROCR (Runtime) API: https://github.com/RadeonOpenCompute/ROCR-Runtime

	= HiHAT overview, PaRSEC, Mar 21, 2017 =		= HiHAT overview, PaRSEC, Mar 21, 2017 =

imported>Cnewburn at 16:36, April 18, 2017

2017-04-18T16:36:12Z

← Older revision		Revision as of 16:36, April 18, 2017
Line 4:		Line 4:
	A link to get back up to the parent page is [[Hierarchical_Heterogeneous_Asynchronous_Tasking \| here]].		A link to get back up to the parent page is [[Hierarchical_Heterogeneous_Asynchronous_Tasking \| here]].
	= Partner interactions, NVIDIA usage models, AMD efforts, Apr 18, 2017 =		= Partner interactions, NVIDIA usage models, AMD efforts, Apr 18, 2017 =
	Attendees included		Attendees (37) included: Wilf, Kath Knobe, Millad Ghane, CJ Newburn, Bill Feiereisen, Dmitry Liakh, Gordon Brown, Jesmin Tithi, Jans Johansen, Jiri Dokulil, John Feo, Kelly Livingston, Max grossman, Mauro Bianco, Oscar Hernandez, Patrick Atkinson, Piotr Luszczek, Ron Brightwell, Ruyman Reyes, Stephen Jones, Stephen Olivier, Sunita Chandrasekaran, Szilard Pall, Wael Elwasif, Ashwin Aji, George Bosilca, John Stone, Benoit Meister, Andrew Lumsdaine, Mike Bauer, Tim

	CJ offered some recent highlights of partner interactions		CJ offered some recent highlights of partner interactions
	* ARM, AMD, IBM engaging		* ARM, AMD, IBM, NVIDIA engaging
	* User story, requirements and app updates		* User story, requirements and app updates
	** Jim Phillips, NAMD; John Stone, VMD; Ronak Buch on Charm++, David Richards on transport Monte Carlo		** Jim Phillips, NAMD; John Stone, VMD; Ronak Buch on Charm++, David Richards on transport Monte Carlo
	** David Keyes, on categories of hierarchical algorithms		** David Keyes, on categories of hierarchical algorithms
			* DHPC++ workshop, Toronto, May 16; will be a talk on HiHAT https://easychair.org/cfp/dhpcc17

	Upcoming HiHAT Mini-Summit		Upcoming HiHAT Mini-Summit
Line 18:		Line 19:
	* NVIDIA interested in HiHAT to broaden access of codes to resources in hetero platforms		* NVIDIA interested in HiHAT to broaden access of codes to resources in hetero platforms
	* Also for AI: deep learning and inference have available tasked-based parallelism		* Also for AI: deep learning and inference have available tasked-based parallelism
			** Offered some background on DNN, RNN
			* As the lower bound on task granularity drops, more task parallelism may be available
			* Two ways to leverage fine-grained tasks better:
			** reduce overheads for actions like invocation and moving data, instigated by CPU and performed on GPU --> lower-overhead Common Layer
			** aggregate tasks in sequences and sub-graphs, that are passed down to target for localized handling --> richer tasking abstractions

	Teaser on relevant AMD efforts, Ashwin Aji, AMD		Teaser on relevant AMD efforts, Ashwin Aji, AMD

imported>Cnewburn at 15:32, April 18, 2017

2017-04-18T15:32:31Z

← Older revision		Revision as of 15:32, April 18, 2017
Line 1:		Line 1:
	This wiki page is used to keep minutes from phone and face to face meetings on the topic of usage models, user stories, and applications for heterogeneous hierarchical asynchronous tasking. Most recent meetings are listed on top.		This wiki page is used to keep minutes from phone and face to face meetings on the topic of usage models, user stories, and applications for heterogeneous hierarchical asynchronous tasking. Most recent meetings are listed on top.
			See '''[Presentations]''' for the ongoing agenda for monthly meetings and the materials that were posted.

			A link to get back up to the parent page is [[Hierarchical_Heterogeneous_Asynchronous_Tasking \| here]].
			= Partner interactions, NVIDIA usage models, AMD efforts, Apr 18, 2017 =
			Attendees included

			CJ offered some recent highlights of partner interactions
			* ARM, AMD, IBM engaging
			* User story, requirements and app updates
			** Jim Phillips, NAMD; John Stone, VMD; Ronak Buch on Charm++, David Richards on transport Monte Carlo
			** David Keyes, on categories of hierarchical algorithms

			Upcoming HiHAT Mini-Summit
			* See info [[HiHAT_First_Mini-Summit \| here]]

			Teaser on NVIDIA usage models, Stephen Jones, NVIDIA
			* NVIDIA interested in HiHAT to broaden access of codes to resources in hetero platforms
			* Also for AI: deep learning and inference have available tasked-based parallelism

			Teaser on relevant AMD efforts, Ashwin Aji, AMD
			* ROCm and ATMI

	= HiHAT overview, PaRSEC, Mar 21, 2017 =		= HiHAT overview, PaRSEC, Mar 21, 2017 =

imported>Cnewburn at 16:58, March 21, 2017

2017-03-21T16:58:33Z

← Older revision		Revision as of 16:58, March 21, 2017
Line 1:		Line 1:
	This wiki page is used to keep minutes from phone and face to face meetings on the topic of usage models, user stories, and applications for heterogeneous hierarchical asynchronous tasking. Most recent meetings are listed on top.		This wiki page is used to keep minutes from phone and face to face meetings on the topic of usage models, user stories, and applications for heterogeneous hierarchical asynchronous tasking. Most recent meetings are listed on top.

			= HiHAT overview, PaRSEC, Mar 21, 2017 =
			Attendees included Wilf Pinfold, Benoit Meister, Patrick Atkinson, Schumann, George Bosilca, Piotr Luszczek, JimPhillips, Stephen Olivier, Max Grossman, Bill Feiereisen, Dmitry Liakh, Wael Elwasif, Jiri Dokulil, Gordon Brown, John Stone, Andrew Lumsdaine, Thomas Herault, Ronak Buch, Ashwin Aji, Bala Seshasayee, Michael Garland, Damien Genet, Aurelien Bouteiller, Oscar Hernandez, PSZ - Paul Szillard?, Timo (Blue Brain), Kelly Livingston, Antonio Tumeo, CJ, several more

			CJ gave a [[media:HiHAT.pdf\|HiHAT Overview]]
			* Progress in funding, e.g. from US government and vendors
			* Several posts to web, including from PASC, Charm++, VMD, Habanero tasking micro-benchmark suite
			* Upcoming report out on progress at GTC, morning of May 9 in San Jose
			** Usage models and requirements
			** Reveal initial progress on prioritized HiHAT interface design
			* Highlighted SW architecture of HiHAT, especially regarding pluggable modules, user layer with target-specific decision making with ease of use, and common layer that dispatches to target-specific implementations of actions
			* Call for more participation in identifying prioritized functionality of HiHAT to leverage, specific requirements and interfaces

			George Bosilca of U Tennessee gave an overview of [[media:HiHat - PaRSEC presentation short.pdf\|PaRSEC interaction with HiHAT]]
			* Data-centric programming environment based on async tasks executing on a hetero distributed environment
			* Offers a domain-specific language interface
			* Delivers good performance and scalability
			* SW architecture is based on modular component architecture of Open MPI, so it's quite amenable to plugging in HiHAT implementations for some of its functionality.
			* Prioritized wish list
			** Portable and efficient API/library for accelerator support - data movement, tasks
			** Portable, efficient and inter-operable communication library (UCX, libFabric, …)
			*** Moving away from MPI will require an efficient datatype engine
			*** Also supported by rest of the software stack (for interoperability)
			** Resource management/allocation system
			*** PaRSEC supports dynamic resource provisioning, but we need a portable system to bridge the gap between different programming runtimes
			** Memory allocator: thread safe, runtime defined properties, arenas (with and without sbrk). (memkind?)
			** Generic profiling system, tools integration
			** Task-based debugger and performance analysis

			Items for potential discussion and investigation
			* Enumeration - look at interaction with HWLOC
			* Dealing with unstructured data and data types
			* Data versioning
			* Serialized streams and subsequences of actions; may want cancellation
			* Resilience - detection, propagation
			* Interfaces for data movement, how that relates to MPI, collectives


	= OCR Review, Feb. 21, 2017 =		= OCR Review, Feb. 21, 2017 =

imported>Wpinfold at 08:11, February 22, 2017

2017-02-22T08:11:53Z

← Older revision		Revision as of 08:11, February 22, 2017
Line 35:		Line 35:
	** Upcoming talks will look at the apps/algos which will be layered on top of HiHAT.		** Upcoming talks will look at the apps/algos which will be layered on top of HiHAT.
	** Lots of good work in progress - appreciate people contributing and sharing		** Lots of good work in progress - appreciate people contributing and sharing
	* Michael Wong: One thing he's looking at is developing heterogeneous C++. If the group is interested he can send out some information about that. Also going to be running a workshop on ISO C/C++ and other high level heterogeneous C++ programming models [~~[media:~~http://sycl.tech/distributed-heterogeneous-programming-in-c-cpp-dhpccpp17.html\|here]].		* Michael Wong: One thing he's looking at is developing heterogeneous C++. If the group is interested he can send out some information about that. Also going to be running a workshop on ISO C/C++ and other high level heterogeneous C++ programming models [http://sycl.tech/distributed-heterogeneous-programming-in-c-cpp-dhpccpp17.html here].
	* CJ: Want to look at these things and decide "would these be called BY HiHAT, or built on top of HiHAT?"		* CJ: Want to look at these things and decide "would these be called BY HiHAT, or built on top of HiHAT?"
	* MW: Do have models which can build on top of HiHAT. Can have discussion at a later meeting.		* MW: Do have models which can build on top of HiHAT. Can have discussion at a later meeting.