Communications - Revision history

imported>Mhall at 23:05, May 20, 2014

2014-05-20T23:05:38Z

← Older revision		Revision as of 23:05, May 20, 2014
Line 26:		Line 26:
	\|dynaxanswer=SWARM uses split-phase asynchronous communications operations with separate setup and callback phases, with a common co-/subroutine-call protocol on top of this. Method calls can easily be relocated by SWARM, and within this infrastructure we additionally provide for split-phase transfers and collective operations. Non-blocking one-sided communication is the only real need, though secondary features (such as remote atomics) might be beneficial.		\|dynaxanswer=SWARM uses split-phase asynchronous communications operations with separate setup and callback phases, with a common co-/subroutine-call protocol on top of this. Method calls can easily be relocated by SWARM, and within this infrastructure we additionally provide for split-phase transfers and collective operations. Non-blocking one-sided communication is the only real need, though secondary features (such as remote atomics) might be beneficial.

	\|xtuneanswer=		\|xtuneanswer=X-TUNE is primarily focused on introducing thread-level parallelism and synchronization, and is relying on programmers or other tools to manage communication. Support in X-TUNE could be used to perform autotuning of communication primitives, but this is beyond the scope of our activities.

	\|gvranswer=		\|gvranswer=
Line 106:		Line 106:
	\|dynaxanswer=The DynAX project will focus on Resilience in year three. As such, it's not yet clear what SWARM's reliability requirements from the communication layer will be. We expect error recovery to occur at the level of the application kernel, by using containment domains to restart a failed tile operation, or algorithmic iteration, or application kernel when the error is detected. The success of this will hinge on whether errors can be reliably detected, and reliably acted upon.		\|dynaxanswer=The DynAX project will focus on Resilience in year three. As such, it's not yet clear what SWARM's reliability requirements from the communication layer will be. We expect error recovery to occur at the level of the application kernel, by using containment domains to restart a failed tile operation, or algorithmic iteration, or application kernel when the error is detected. The success of this will hinge on whether errors can be reliably detected, and reliably acted upon.

	\|xtuneanswer=		\|xtuneanswer=N/A

	\|gvranswer=		\|gvranswer=
Line 133:		Line 133:
	As part of PEDAL (Power Efficient Data Abstraction Layer), we are also developing an additional SW layer that encapsulates data composites. This process assigns specific layouts, transformations and operators to the composites that can be used advantageously to reduce power and energy costs. A similar process will we applicable to resiliency as well.		As part of PEDAL (Power Efficient Data Abstraction Layer), we are also developing an additional SW layer that encapsulates data composites. This process assigns specific layouts, transformations and operators to the composites that can be used advantageously to reduce power and energy costs. A similar process will we applicable to resiliency as well.

	\|xtuneanswer=		\|xtuneanswer=N/A

	\|gvranswer=		\|gvranswer=
Line 160:		Line 160:
	\|dynaxanswer=Vendor mechanisms are helpful, but not necessary, for ensuring the correctness and timeliness of low-level data transfers. SWARM itself uses a higher-level flow-control mechanism based on codelet completion (i.e., using callbacks as a basis for issuing ACKs). SWARM also performs load-balancing on work and data to help minimize congestion and contention.		\|dynaxanswer=Vendor mechanisms are helpful, but not necessary, for ensuring the correctness and timeliness of low-level data transfers. SWARM itself uses a higher-level flow-control mechanism based on codelet completion (i.e., using callbacks as a basis for issuing ACKs). SWARM also performs load-balancing on work and data to help minimize congestion and contention.

	\|xtuneanswer=		\|xtuneanswer=N/A

	\|gvranswer=		\|gvranswer=

imported>Milind at 16:38, May 20, 2014

2014-05-20T16:38:58Z

← Older revision		Revision as of 16:38, May 20, 2014
Line 32:		Line 32:
	\|corvetteanswer=		\|corvetteanswer=

	\|sleecanswer=		\|sleecanswer=N/A

	\|piperanswer=Communication will be out of band and need to be isolated, emphasis on streaming communication		\|piperanswer=Communication will be out of band and need to be isolated, emphasis on streaming communication
Line 57:		Line 57:
	\|corvetteanswer=		\|corvetteanswer=

	\|sleecanswer=~~N/A~~		\|sleecanswer=Computation methods should express their dependences so that SLEEC's runtime(s) can manage communication within a heterogeneous node.

	\|piperanswer=N/A		\|piperanswer=N/A

imported>Admin at 05:18, May 20, 2014

2014-05-20T05:18:46Z

Show changes

imported>Admin at 05:17, May 20, 2014

2014-05-20T05:17:10Z

Show changes

imported>Rlawton at 21:15, May 13, 2014

2014-05-13T21:15:58Z

← Older revision		Revision as of 21:15, May 13, 2014
Line 76:		Line 76:
	\| Not a topic of research for D-TEC.		\| Not a topic of research for D-TEC.
	\|Data transfers will occupy a large portion of the energy budget, so minimizing the need for data movement will greatly improve energy consumption. This can be done by ensuring that, whenever possible, work is forwarded to the data and not vice versa. SWARM's codelet model ensures that this is quasi-transparent to the programmer, although the runtime must obviously perform the work-forwarding and whatever data relocations are needed itself. Hints from the compiler or application programmer/tuner can assist the runtime in this and further decrease energy consumption.		\|Data transfers will occupy a large portion of the energy budget, so minimizing the need for data movement will greatly improve energy consumption. This can be done by ensuring that, whenever possible, work is forwarded to the data and not vice versa. SWARM's codelet model ensures that this is quasi-transparent to the programmer, although the runtime must obviously perform the work-forwarding and whatever data relocations are needed itself. Hints from the compiler or application programmer/tuner can assist the runtime in this and further decrease energy consumption.

			As part of PEDAL (Power Efficient Data Abstraction Layer), we are also developing an additional SW layer that encapsulates data composites. This process assigns specific layouts, transformations and operators to the composites that can be used advantageously to reduce power and energy costs. A similar process will we applicable to resiliency as well.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)

imported>PaulHargrove at 16:02, May 12, 2014

2014-05-12T16:02:33Z

← Older revision		Revision as of 16:02, May 12, 2014
Line 32:		Line 32:
	\|''It is important for performance portability that most communication related codes comprise invariants that will always be true. Aggregation, routing, order sensitivity, time to arrival, and error management should be transparent to the user. Destinations, in most cases, should be relative to placement of first class objects for adaptive placement and routing. Scheduling should tolerate asynchrony uncertainty of message delivery without forfeit of performance assuming sufficient parallelism.''		\|''It is important for performance portability that most communication related codes comprise invariants that will always be true. Aggregation, routing, order sensitivity, time to arrival, and error management should be transparent to the user. Destinations, in most cases, should be relative to placement of first class objects for adaptive placement and routing. Scheduling should tolerate asynchrony uncertainty of message delivery without forfeit of performance assuming sufficient parallelism.''
	\|(TG)		\|(TG)
	\|DEGAS is extending the GASNet APIs to produce GASNet-EX. GASNet-EX is designed to '''support''' the computation model rather than dictate it. Unlike the current GASNet, GASNet-EX ~~treats~~ threads as first-class entities (as in the MPI endpoints proposal), allowing efficient mapping of non-SPMD execution models, e.g. Habanero, that are impractical or inefficient today.		\|DEGAS is extending the GASNet APIs to produce GASNet-EX. GASNet-EX is designed to '''support''' the computation model rather than dictate it. Unlike the current GASNet, GASNet-EX allows (but does not require) treatment of threads as first-class entities (as in the MPI endpoints proposal), allowing efficient mapping of non-SPMD execution models, e.g. Habanero, that are impractical or inefficient today.
	\| It is not clear to us that tight integration of communication libraries and the computation model is needed to support non SPMD execution. The X10/APGAS runtime supports non-SPMD execution at scale while maintaining a fairly strict separation between the communication layer (X10RT) and the computational model. X10RT provides basic active message facilities, but all higher-level computational model concerns are handled above the X10RT layer of the runtime. However, there are certainly opportunities to optimize some of the small "control" messages sent over the X10RT transport by the APGAS runtime layer by off-loading pieces of runtime logic into message handlers that could run directly within the network software/hardware. Pushing this function down simply requires the network layer to allow execution of user-provided handlers, not a true integration of the computation model into the communication library.		\| It is not clear to us that tight integration of communication libraries and the computation model is needed to support non SPMD execution. The X10/APGAS runtime supports non-SPMD execution at scale while maintaining a fairly strict separation between the communication layer (X10RT) and the computational model. X10RT provides basic active message facilities, but all higher-level computational model concerns are handled above the X10RT layer of the runtime. However, there are certainly opportunities to optimize some of the small "control" messages sent over the X10RT transport by the APGAS runtime layer by off-loading pieces of runtime logic into message handlers that could run directly within the network software/hardware. Pushing this function down simply requires the network layer to allow execution of user-provided handlers, not a true integration of the computation model into the communication library.
	\|SWARM's codelet model can be used to effect split-phase co-/subroutine calls, whether or not there are networking features present. Applications can control the routing of these calls and their associated data explicitly, but this is not necessary unless higher-level partitioning schemes are being used. This scheme allows any computation encompassed by one or more codelets to be relocated, allowing both data and computation to be relocated transparently and enabling scaling from a single hardware thread to thousands of nodes without rewriting the application.		\|SWARM's codelet model can be used to effect split-phase co-/subroutine calls, whether or not there are networking features present. Applications can control the routing of these calls and their associated data explicitly, but this is not necessary unless higher-level partitioning schemes are being used. This scheme allows any computation encompassed by one or more codelets to be relocated, allowing both data and computation to be relocated transparently and enabling scaling from a single hardware thread to thousands of nodes without rewriting the application.

imported>Rlawton at 15:20, May 12, 2014

2014-05-12T15:20:56Z

← Older revision		Revision as of 15:20, May 12, 2014
Line 22:		Line 22:
	The GASNet-EX communications library provides Active Messages (AM), one-sided data-movement and collectives as its primary communications primitives. Secondary primitives, such as atomics, may be emulated via AM or implemented through native hardware when available. The programming models in the DEGAS project use these primitives for many purposes. The Active Message primitives support the asynchronous remote invocation operations present in the Habanero and UPC++ efforts, while atomics will provide efficient point-to-point synchronization.		The GASNet-EX communications library provides Active Messages (AM), one-sided data-movement and collectives as its primary communications primitives. Secondary primitives, such as atomics, may be emulated via AM or implemented through native hardware when available. The programming models in the DEGAS project use these primitives for many purposes. The Active Message primitives support the asynchronous remote invocation operations present in the Habanero and UPC++ efforts, while atomics will provide efficient point-to-point synchronization.
	\|The primary focus is computation via asynchronous tasks. The primary communication primitive is (reliably delivered) fire-and-forget active messages. Higher level behavior (finish, at) is synthesized by the APGAS runtime ontop of the active message primitive. However, for performance at scale we recognize the importance of additional primitives: both non-blocking collectives and one-sided asynchronous RDMA's for point-to-point bulk data transfer.		\|The primary focus is computation via asynchronous tasks. The primary communication primitive is (reliably delivered) fire-and-forget active messages. Higher level behavior (finish, at) is synthesized by the APGAS runtime ontop of the active message primitive. However, for performance at scale we recognize the importance of additional primitives: both non-blocking collectives and one-sided asynchronous RDMA's for point-to-point bulk data transfer.
	\|(~~DynAX~~)		\|SWARM uses split-phase asynchronous communications operations with separate setup and callback phases, with a common co-/subroutine-call protocol on top of this. Method calls can easily be relocated by SWARM, and within this infrastructure we additionally provide for split-phase transfers and collective operations. Non-blocking one-sided communication is the only real need, though secondary features (such as remote atomics) might be beneficial.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 34:		Line 34:
	\|DEGAS is extending the GASNet APIs to produce GASNet-EX. GASNet-EX is designed to '''support''' the computation model rather than dictate it. Unlike the current GASNet, GASNet-EX treats threads as first-class entities (as in the MPI endpoints proposal), allowing efficient mapping of non-SPMD execution models, e.g. Habanero, that are impractical or inefficient today.		\|DEGAS is extending the GASNet APIs to produce GASNet-EX. GASNet-EX is designed to '''support''' the computation model rather than dictate it. Unlike the current GASNet, GASNet-EX treats threads as first-class entities (as in the MPI endpoints proposal), allowing efficient mapping of non-SPMD execution models, e.g. Habanero, that are impractical or inefficient today.
	\| It is not clear to us that tight integration of communication libraries and the computation model is needed to support non SPMD execution. The X10/APGAS runtime supports non-SPMD execution at scale while maintaining a fairly strict separation between the communication layer (X10RT) and the computational model. X10RT provides basic active message facilities, but all higher-level computational model concerns are handled above the X10RT layer of the runtime. However, there are certainly opportunities to optimize some of the small "control" messages sent over the X10RT transport by the APGAS runtime layer by off-loading pieces of runtime logic into message handlers that could run directly within the network software/hardware. Pushing this function down simply requires the network layer to allow execution of user-provided handlers, not a true integration of the computation model into the communication library.		\| It is not clear to us that tight integration of communication libraries and the computation model is needed to support non SPMD execution. The X10/APGAS runtime supports non-SPMD execution at scale while maintaining a fairly strict separation between the communication layer (X10RT) and the computational model. X10RT provides basic active message facilities, but all higher-level computational model concerns are handled above the X10RT layer of the runtime. However, there are certainly opportunities to optimize some of the small "control" messages sent over the X10RT transport by the APGAS runtime layer by off-loading pieces of runtime logic into message handlers that could run directly within the network software/hardware. Pushing this function down simply requires the network layer to allow execution of user-provided handlers, not a true integration of the computation model into the communication library.
	\|~~(DynAX)~~		\|SWARM's codelet model can be used to effect split-phase co-/subroutine calls, whether or not there are networking features present. Applications can control the routing of these calls and their associated data explicitly, but this is not necessary unless higher-level partitioning schemes are being used. This scheme allows any computation encompassed by one or more codelets to be relocated, allowing both data and computation to be relocated transparently and enabling scaling from a single hardware thread to thousands of nodes without rewriting the application.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 49:		Line 49:
	Primary metrics for communications runtime include latency of short messages, bandwidth of large messages, and communication/computation overlap opportunity during long-latency operations. Reduction of energy is a metric for the stack as a whole and may be more dependent on '''avoiding''' communication than on optimizing it (see also energy-related question).		Primary metrics for communications runtime include latency of short messages, bandwidth of large messages, and communication/computation overlap opportunity during long-latency operations. Reduction of energy is a metric for the stack as a whole and may be more dependent on '''avoiding''' communication than on optimizing it (see also energy-related question).
	\| Under the assumption that the communication layer is not tightly integrated with the computation model, the scope of transparent optimization seems limited to optimizing the flow of traffic within the network. Perhaps also providing performance diagnostics and control points to higher-levels of the runtime to enable them to optimize communication behavior. Optimizations need to be planned/managed at a level of the stack that has sufficient scope to make good decisions.		\| Under the assumption that the communication layer is not tightly integrated with the computation model, the scope of transparent optimization seems limited to optimizing the flow of traffic within the network. Perhaps also providing performance diagnostics and control points to higher-levels of the runtime to enable them to optimize communication behavior. Optimizations need to be planned/managed at a level of the stack that has sufficient scope to make good decisions.
	\|(~~DynAX~~)		\|A communications layer should ideally be able to load-balance both work and data without application involvement, using optional application-provided placement hints to assist in the process. Compilers should deal more with transforming higher-level language features like data types and method calls into SWARM constructs, and although compilers may generate hints, this will likely have to be the responsibility of the developer and tuner. The primary (external) metrics used are time to application completion and energy cost.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 63:		Line 63:
	The communications library supports each resilience mechanism in appropriate ways. For rollback-recovery, GASNet-EX must include a mechanism to capture a consistent state, a significantly more challenging a problem with one-sided communication than in a message-passing system, especially if one does not wish to quiesce all application communications for a consistent checkpoint. For Containment Domains, GASNet-EX must run through communications failures by reacting (not aborting), by notifying other runtime components, by enabling these components to take appropriate actions, and by preventing resource leaks associated with (for instance) now-unreachable peers.		The communications library supports each resilience mechanism in appropriate ways. For rollback-recovery, GASNet-EX must include a mechanism to capture a consistent state, a significantly more challenging a problem with one-sided communication than in a message-passing system, especially if one does not wish to quiesce all application communications for a consistent checkpoint. For Containment Domains, GASNet-EX must run through communications failures by reacting (not aborting), by notifying other runtime components, by enabling these components to take appropriate actions, and by preventing resource leaks associated with (for instance) now-unreachable peers.
	\| This is not an area of research for D-TEC. We are assuming the low-level communication libraries will (at least as visible by our layer) operate correctly or report faults when it does not. Any faults reported by the underlying communication library will be reflected up to higher-levels of the runtime stack.		\| This is not an area of research for D-TEC. We are assuming the low-level communication libraries will (at least as visible by our layer) operate correctly or report faults when it does not. Any faults reported by the underlying communication library will be reflected up to higher-levels of the runtime stack.
	\|(DynAX)		\|The DynAX project will focus on Resilience in year three. As such, it's not yet clear what SWARM's reliability requirements from the communication layer will be. We expect error recovery to occur at the level of the application kernel, by using containment domains to restart a failed tile operation, or algorithmic iteration, or application kernel when the error is detected. The success of this will hinge on whether errors can be reliably detected, and reliably acted upon.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 75:		Line 75:
	\| If/when applications become blocked waiting for communications to complete, one should consider energy-aware mechanisms for blocking. Other than that, most mechanisms for energy reduction with respect to communication are also effective for reducing time-to-solution and are likely to be studied in that context (where the metric is easier to measure).		\| If/when applications become blocked waiting for communications to complete, one should consider energy-aware mechanisms for blocking. Other than that, most mechanisms for energy reduction with respect to communication are also effective for reducing time-to-solution and are likely to be studied in that context (where the metric is easier to measure).
	\| Not a topic of research for D-TEC.		\| Not a topic of research for D-TEC.
	\|~~(DynAX)~~		\|Data transfers will occupy a large portion of the energy budget, so minimizing the need for data movement will greatly improve energy consumption. This can be done by ensuring that, whenever possible, work is forwarded to the data and not vice versa. SWARM's codelet model ensures that this is quasi-transparent to the programmer, although the runtime must obviously perform the work-forwarding and whatever data relocations are needed itself. Hints from the compiler or application programmer/tuner can assist the runtime in this and further decrease energy consumption.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 89:		Line 89:
	Ideally, the vendor systems would provide some degree of congestion management. This would use information not necessarily available to the communications runtime, e.g. static information about their network, dynamic information about application traffic, and traffic from ''other'' jobs. However, compilers and runtime components with "macro" communications behaviors, i.e. collectives or other structured communications, could potentially inform the communications layer about near-future communications, where this information can be used to build congestion-avoiding communications schedules. These scheduling approaches can be greatly enhanced if the vendors expose information about current network conditions, particularly for networks where multiple jobs share the same links.		Ideally, the vendor systems would provide some degree of congestion management. This would use information not necessarily available to the communications runtime, e.g. static information about their network, dynamic information about application traffic, and traffic from ''other'' jobs. However, compilers and runtime components with "macro" communications behaviors, i.e. collectives or other structured communications, could potentially inform the communications layer about near-future communications, where this information can be used to build congestion-avoiding communications schedules. These scheduling approaches can be greatly enhanced if the vendors expose information about current network conditions, particularly for networks where multiple jobs share the same links.
	\| Higher levels should focus on task & data placement to increase locality and reduce redundant communication. Placement should also be aware of network topology and optimize towards keeping frequently communicating tasks in the same "neighborhood" when possible. Micro-optimization of routing, congestion management, and flow control are probably most effectively handled by system/vendor mechanisms since it may require detailed understanding of the internals of network software/hardware and the available dynamic information.		\| Higher levels should focus on task & data placement to increase locality and reduce redundant communication. Placement should also be aware of network topology and optimize towards keeping frequently communicating tasks in the same "neighborhood" when possible. Micro-optimization of routing, congestion management, and flow control are probably most effectively handled by system/vendor mechanisms since it may require detailed understanding of the internals of network software/hardware and the available dynamic information.
	\|(~~DynAX~~)		\|Vendor mechanisms are helpful, but not necessary, for ensuring the correctness and timeliness of low-level data transfers. SWARM itself uses a higher-level flow-control mechanism based on codelet completion (i.e., using callbacks as a basis for issuing ACKs). SWARM also performs load-balancing on work and data to help minimize congestion and contention.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)

imported>PaulHargrove at 00:13, May 10, 2014

2014-05-10T00:13:55Z

← Older revision		Revision as of 00:13, May 10, 2014
Line 87:		Line 87:
	\|As others have observed, the first line of defense against congestion is intelligent placement of tasks and their data. This is the domain of the tasking runtime and the application author.		\|As others have observed, the first line of defense against congestion is intelligent placement of tasks and their data. This is the domain of the tasking runtime and the application author.

	Ideally, the vendor systems would provide some degree of congestion management. This would use information not necessarily available to the communications runtime, e.g. static information about their network, dynamic information about application traffic, and traffic from ''other'' jobs. However, compilers and runtime components with "macro" communications behaviors, i.e. collectives or other structured communications, could potentially inform the communications layer about near-future communications, where this information can be used to build congestion-avoiding communications schedules. These scheduling approaches can be greatly enhanced if the vendors ~~provide~~ information about current network conditions, particularly for networks where multiple jobs share the same links.		Ideally, the vendor systems would provide some degree of congestion management. This would use information not necessarily available to the communications runtime, e.g. static information about their network, dynamic information about application traffic, and traffic from ''other'' jobs. However, compilers and runtime components with "macro" communications behaviors, i.e. collectives or other structured communications, could potentially inform the communications layer about near-future communications, where this information can be used to build congestion-avoiding communications schedules. These scheduling approaches can be greatly enhanced if the vendors expose information about current network conditions, particularly for networks where multiple jobs share the same links.
	\| Higher levels should focus on task & data placement to increase locality and reduce redundant communication. Placement should also be aware of network topology and optimize towards keeping frequently communicating tasks in the same "neighborhood" when possible. Micro-optimization of routing, congestion management, and flow control are probably most effectively handled by system/vendor mechanisms since it may require detailed understanding of the internals of network software/hardware and the available dynamic information.		\| Higher levels should focus on task & data placement to increase locality and reduce redundant communication. Placement should also be aware of network topology and optimize towards keeping frequently communicating tasks in the same "neighborhood" when possible. Micro-optimization of routing, congestion management, and flow control are probably most effectively handled by system/vendor mechanisms since it may require detailed understanding of the internals of network software/hardware and the available dynamic information.
	\|(DynAX)		\|(DynAX)

imported>PaulHargrove at 00:12, May 10, 2014

2014-05-10T00:12:36Z

← Older revision		Revision as of 00:12, May 10, 2014
Line 61:		Line 61:
	\|DEGAS is pursuing a hybrid approach to resilience which consists of both backward recovery (rollback-recovery of state via checkpoint/restart) and forward recovery (recompute or re-communicate faulty state via Containment Domains), working together in the same run. The ability of Containment Domains to isolate faults and to perform most recovery ''locally'' is ideal for most "soft errors", while the use of rollback-recovery is appropriate to hard node crashes. The combination of the two not only reduces the frequency of checkpoints required to provide effective protection, but also limits the type of errors that an application programmer must tolerate. Further, our approach allows the scope of rollback-recovery to be limited to subsets of the nodes and, in some cases, only the faulty nodes need to perform recovery.		\|DEGAS is pursuing a hybrid approach to resilience which consists of both backward recovery (rollback-recovery of state via checkpoint/restart) and forward recovery (recompute or re-communicate faulty state via Containment Domains), working together in the same run. The ability of Containment Domains to isolate faults and to perform most recovery ''locally'' is ideal for most "soft errors", while the use of rollback-recovery is appropriate to hard node crashes. The combination of the two not only reduces the frequency of checkpoints required to provide effective protection, but also limits the type of errors that an application programmer must tolerate. Further, our approach allows the scope of rollback-recovery to be limited to subsets of the nodes and, in some cases, only the faulty nodes need to perform recovery.

	The communications library supports each resilience mechanism in appropriate ways. For rollback-recovery, GASNet-EX must include a mechanism to capture a consistent state, a significantly more challenging a problem with one-sided communication than in a message-passing system, especially if one does not wish to quiesce all application communications for a consistent checkpoint. For Containment Domains, GASNet-EX must run through communications failures by reacting (not aborting), by notifying other runtime components, by ~~allowing~~ these ~~compenents~~ to take appropriate actions, and by preventing resource leaks associated with (for instance) now-unreachable peers.		The communications library supports each resilience mechanism in appropriate ways. For rollback-recovery, GASNet-EX must include a mechanism to capture a consistent state, a significantly more challenging a problem with one-sided communication than in a message-passing system, especially if one does not wish to quiesce all application communications for a consistent checkpoint. For Containment Domains, GASNet-EX must run through communications failures by reacting (not aborting), by notifying other runtime components, by enabling these components to take appropriate actions, and by preventing resource leaks associated with (for instance) now-unreachable peers.
	\| This is not an area of research for D-TEC. We are assuming the low-level communication libraries will (at least as visible by our layer) operate correctly or report faults when it does not. Any faults reported by the underlying communication library will be reflected up to higher-levels of the runtime stack.		\| This is not an area of research for D-TEC. We are assuming the low-level communication libraries will (at least as visible by our layer) operate correctly or report faults when it does not. Any faults reported by the underlying communication library will be reflected up to higher-levels of the runtime stack.
	\|(DynAX)		\|(DynAX)

imported>PaulHargrove at 00:11, May 10, 2014

2014-05-10T00:11:25Z

← Older revision		Revision as of 00:11, May 10, 2014
Line 47:		Line 47:
	\| Communications libraries and neighboring runtime layers should be responsible only for '''dynamic''' optimization of communication. Examples of such optimizations include: aggregation of messages with the same destination, scheduling multiple links, and injection control for congestion avoidance. Compilers or application developers should be responsible for '''static''' optimizations such as communication avoidance, hot-spot elimination, etc.		\| Communications libraries and neighboring runtime layers should be responsible only for '''dynamic''' optimization of communication. Examples of such optimizations include: aggregation of messages with the same destination, scheduling multiple links, and injection control for congestion avoidance. Compilers or application developers should be responsible for '''static''' optimizations such as communication avoidance, hot-spot elimination, etc.

	Primary metrics for communications runtime include latency of short messages, bandwidth of large messages, and ~~communications~~/computation overlap opportunity during long-latency operations. Reduction of energy is a metric for the stack as a whole and may be more dependent on '''avoiding''' communication than on optimizing it (see also energy-related question).		Primary metrics for communications runtime include latency of short messages, bandwidth of large messages, and communication/computation overlap opportunity during long-latency operations. Reduction of energy is a metric for the stack as a whole and may be more dependent on '''avoiding''' communication than on optimizing it (see also energy-related question).
	\| Under the assumption that the communication layer is not tightly integrated with the computation model, the scope of transparent optimization seems limited to optimizing the flow of traffic within the network. Perhaps also providing performance diagnostics and control points to higher-levels of the runtime to enable them to optimize communication behavior. Optimizations need to be planned/managed at a level of the stack that has sufficient scope to make good decisions.		\| Under the assumption that the communication layer is not tightly integrated with the computation model, the scope of transparent optimization seems limited to optimizing the flow of traffic within the network. Perhaps also providing performance diagnostics and control points to higher-levels of the runtime to enable them to optimize communication behavior. Optimizations need to be planned/managed at a level of the stack that has sufficient scope to make good decisions.
	\|(DynAX)		\|(DynAX)