Operating Systems - Revision history

imported>Admin at 05:01, May 20, 2014

2014-05-20T05:01:08Z

imported>Admin at 04:56, May 20, 2014

2014-05-20T04:56:01Z

← Older revision		Revision as of 04:56, May 20, 2014
Line 9:		Line 9:
	* [[#table3\|What energy/power and reliability APIs do you expect from the OS/R?]]		* [[#table3\|What energy/power and reliability APIs do you expect from the OS/R?]]
	* [[#table4\|Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?]]		* [[#table4\|Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?]]
	* [[#table5\|]]		* [[#table5\|What is your model for extreme-scale I/O, and how do you expect the OS/R to support your plans?]]
	* [[#table6\|~~After a bug is discovered~~, do you ~~want automated~~ support ~~to create a simplified test that will reproduce the bug~~?]]		* [[#table6\|Does your project include support for multiple, hierarchical, memory regions? If so, how will it be allocated and managed, by your xstack project or my the OS/R? What APIs do you expect the OS/R to support complex and deep memory?]]

	<div id="table1"></div>{{		<div id="table1"></div>{{

imported>Admin at 04:55, May 20, 2014

2014-05-20T04:55:21Z

Show changes

imported>Rlawton at 13:38, May 12, 2014

2014-05-12T13:38:42Z

← Older revision		Revision as of 13:38, May 12, 2014
Line 21:		Line 21:
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
	\|(~~DynAX~~)		\|The SWARM runtime requires access to hardware threads, memory, and network interconnect(s), whether by system call or direct access. On commodity clusters, SWARM additionally needs access to I/O facilities, such as the POSIX select, read, and write calls.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 33:		Line 33:
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
	\|~~(DynAX)~~		\|SWARM uses codelets to intermediate between hardware cores and function/method calls. The general requirement is for direct allocation of hardware resources. On Linux-like platforms, threads are created and bound to cores at runtime startup; codelets are bound to particular threads only when they're dispatched, unless some more specific binding is arranged for before readying the codelet.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 45:		Line 45:
	\|(DEGAS)		\|(DEGAS)
	\| Virtualization is an orthogonal issue for D-TEC. We do not assume either the presence or absence of virtualization in our work.		\| Virtualization is an orthogonal issue for D-TEC. We do not assume either the presence or absence of virtualization in our work.
	\|~~(DynAX)~~		\|No.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 62:		Line 62:
	- Access to DVFS settings		- Access to DVFS settings

	\|~~(DynAX)~~		\|The ability to dynamically modify power state and clock frequency for subsets of the system, and a mechanism to detect failures, are both very important for meeting power and reliability goals.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 74:		Line 74:
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
	\|~~(DynAX)~~		\|Parallel programs comprise a directed acyclic graph of non-blocking tasks, called "codelets," which form the nodes of the graph, and which produce and consume data, which form the edges of the graph. The SWARM runtime keeps track of when each task's input dependencies are met and when it can run, and where it can run in order to maximize locality to minimize the amount of required data movement. Fundamentally, this only requires the OS or hardware to permit SWARM to allocate hardware cores and memory, and to send data around the system. To reduce runtime overhead, hardware features such as DMA would be beneficial. If such hardware features are protected from direct access by the SWARM runtime, then the OS should expose an API to access them.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 86:		Line 86:
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
	\|~~(DynAX)~~		\|At Exascale, it is expected that network latencies will be very high, and that hardware failure is common. Therefore, at the OS/runtime level, what's needed is an interface for doing asynchronous I/O which is resilient in the face of failures. Further details will be dictated largely by the needs of applications.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)
Line 98:		Line 98:
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
	\|~~(DynAX)~~		\|The SWARM runtime does support distributed memory and deep NUMA machines, through the use of a hierarchy of locale structures which roughly parallel the hardware topology. SWARM requires the ability to allocate the relevant memory regions from the OS.
	\|(X-TUNE)		\|(X-TUNE)
	\|(GVR)		\|(GVR)

imported>Jsstone1 at 03:43, May 12, 2014

2014-05-12T03:43:28Z

← Older revision		Revision as of 03:43, May 12, 2014
Line 71:		Line 71:
	\|'''Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?'''		\|'''Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?'''
	\|Parallel programs are composed through ParalleX Processes interfaces comprising message-driven method instantiation for value/synchronization passing and data object access. OS support for memory address and global name-space is required.		\|Parallel programs are composed through ParalleX Processes interfaces comprising message-driven method instantiation for value/synchronization passing and data object access. OS support for memory address and global name-space is required.
	\|See the [[About_OCR_-_Open_Community_Runtime\|OCR spec]].		\|See the [[Traleika_Glacier#About_OCR_-_Open_Community_Runtime\|OCR spec]].
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)

imported>Jsstone1 at 03:42, May 12, 2014

2014-05-12T03:42:26Z

← Older revision		Revision as of 03:42, May 12, 2014
Line 71:		Line 71:
	\|'''Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?'''		\|'''Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?'''
	\|Parallel programs are composed through ParalleX Processes interfaces comprising message-driven method instantiation for value/synchronization passing and data object access. OS support for memory address and global name-space is required.		\|Parallel programs are composed through ParalleX Processes interfaces comprising message-driven method instantiation for value/synchronization passing and data object access. OS support for memory address and global name-space is required.
	\|See the [[~~href:https://xstackwiki.modelado.org/Traleika_Glacier#~~About_OCR_-_Open_Community_Runtime\|OCR spec]].		\|See the [[About_OCR_-_Open_Community_Runtime\|OCR spec]].
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)

imported>Jsstone1 at 03:41, May 12, 2014

2014-05-12T03:41:28Z

← Older revision		Revision as of 03:41, May 12, 2014
Line 16:		Line 16:
	\| '''PI''' \|\| Ron Brightwell \|\| Shekhar Borkar \|\| Katherine Yelick \|\| Daniel Quinlan \|\| Guang Gao \|\| Mary Hall \|\| Andrew Chien \|\| Koushik Sen \|\| Milind Kulkarni \|\| Martin Schulz		\| '''PI''' \|\| Ron Brightwell \|\| Shekhar Borkar \|\| Katherine Yelick \|\| Daniel Quinlan \|\| Guang Gao \|\| Mary Hall \|\| Andrew Chien \|\| Koushik Sen \|\| Milind Kulkarni \|\| Martin Schulz
	\|- style="vertical-align:top;"		\|- style="vertical-align:top;"
	\|What are the key system calls / features that you need OS/R to support? Examples: exit, read, write, open, close, link, unlink, chdir, time, chmod, clone, uname, execv, etc.		\|'''What are the key system calls / features that you need OS/R to support? Examples: exit, read, write, open, close, link, unlink, chdir, time, chmod, clone, uname, execv, etc.'''
	\|''LXK is developed within XPRESS as an independent lightweight kernel to fully support the HPX runtime system and XPI programming interfaces through the RIOS (runtime interface to OS) protocol.''		\|LXK is developed within XPRESS as an independent lightweight kernel to fully support the HPX runtime system and XPI programming interfaces through the RIOS (runtime interface to OS) protocol.
	\|(TG)		\|We currently require some method to get memory from the system (equivalent of sbrk) and some method of input/output. The requirements will be extended as we see the need but we want to limit the dependence on system calls to make our runtime as general and applicable to as wide a range of targets as possible.
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
Line 28:		Line 28:
	\|sockets, ptrace access, dynamic linking, timer interrupts/signals, access to hardware counters, file I/O		\|sockets, ptrace access, dynamic linking, timer interrupts/signals, access to hardware counters, file I/O
	\|- style="vertical-align:top;"		\|- style="vertical-align:top;"
	\|Does your project implement it's own lightweight thread package, or does it rely on the threads provided by the OS/R? If you implement your own threads, what are the key features that required developing a new implementation? If you don't implement your own thread package, what are the key performance characteristics and APIs needed to support your project?		\|'''Does your project implement it's own lightweight thread package, or does it rely on the threads provided by the OS/R? If you implement your own threads, what are the key features that required developing a new implementation? If you don't implement your own thread package, what are the key performance characteristics and APIs needed to support your project?'''
	\|''HPX provides its own lightweight thread package that relies on heavyweight thread execution by the LXK OS. Features required include threads as first class objects, efficient context switch, application dynamic scheduling policies, message-driven remote thread creation.''		\|HPX provides its own lightweight thread package that relies on heavyweight thread execution by the LXK OS. Features required include threads as first class objects, efficient context switch, application dynamic scheduling policies, message-driven remote thread creation.
	\|(TG)		\|Our runtime can rely on existing thread (we use pthreads for example) frameworks but we do not need them as we really use the existing threading framework to emulate a computing resource.
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
Line 40:		Line 40:
	\|relying on native threads, typically pthreads, which is appropriate		\|relying on native threads, typically pthreads, which is appropriate
	\|- style="vertical-align:top;"		\|- style="vertical-align:top;"
	\|Do you currently view virtualization as a key requirement to support your x-stack project? If so, why?		\|'''Do you currently view virtualization as a key requirement to support your x-stack project? If so, why?'''
	\|''Virtualization is a key requirement for advanced multi-modal use of LXK supporting a diversity of runtime environments including but not limited to conventional legacy codes and future runtime systems like HPX.''		\|Virtualization is a key requirement for advanced multi-modal use of LXK supporting a diversity of runtime environments including but not limited to conventional legacy codes and future runtime systems like HPX.
	\|~~(TG)~~		\|No.
	\|(DEGAS)		\|(DEGAS)
	\| Virtualization is an orthogonal issue for D-TEC. We do not assume either the presence or absence of virtualization in our work.		\| Virtualization is an orthogonal issue for D-TEC. We do not assume either the presence or absence of virtualization in our work.
Line 52:		Line 52:
	\|No		\|No
	\|- style="vertical-align:top;"		\|- style="vertical-align:top;"
	\|What energy/power and reliability APIs do you expect from the OS/R?		\|'''What energy/power and reliability APIs do you expect from the OS/R?'''
	\|''While XPRESS is not explicitly addressing energy and reliability issues, under related support it does incorporate the microcheckpointing compute-validate-commit cycle for reliability and the side-path energy suppression strategy for power reduction. Access to faults detection from hardware and OS is required as is measurements of power and thermal conditions on per socket basis. Control of processor core operation and clocks are required.''		\|While XPRESS is not explicitly addressing energy and reliability issues, under related support it does incorporate the microcheckpointing compute-validate-commit cycle for reliability and the side-path energy suppression strategy for power reduction. Access to faults detection from hardware and OS is required as is measurements of power and thermal conditions on per socket basis. Control of processor core operation and clocks are required.
	\|(TG)		\|We hope that this will all be managed inside the runtime layer and that our EDT and DB programming model will enable the user to be unaware of it. We may, however, allow for some hints from higher level tools (for example having a compiler expose multiple versions of an EDT with differing energy envelopes). The runtime will rely on the underlying system-software/OS to provide introspection capabilities as well as "knobs" to turn (mostly voltage and frequency regulation as well as fine-grained on/off).
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
Line 69:		Line 69:
	\|Power: Power measurements at varying granularity using processor internal counters and external sensors, for autotuning access to setting DVFS levels or power capping; Resilience: access to fault notification (for corrected and non-corrected errors)		\|Power: Power measurements at varying granularity using processor internal counters and external sensors, for autotuning access to setting DVFS levels or power capping; Resilience: access to fault notification (for corrected and non-corrected errors)
	\|- style="vertical-align:top;"		\|- style="vertical-align:top;"
	\|Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?		\|'''Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?'''
	\|''Parallel programs are composed through ParalleX Processes interfaces comprising message-driven method instantiation for value/synchronization passing and data object access. OS support for memory address and global name-space is required.''		\|Parallel programs are composed through ParalleX Processes interfaces comprising message-driven method instantiation for value/synchronization passing and data object access. OS support for memory address and global name-space is required.
	\|~~(TG)~~		\|See the [[href:https://xstackwiki.modelado.org/Traleika_Glacier#About_OCR_-_Open_Community_Runtime\|OCR spec]].
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
Line 81:		Line 81:
	\|N/A		\|N/A
	\|- style="vertical-align:top;"		\|- style="vertical-align:top;"
	\|What is your model for extreme-scale I/O, and how do you expect the OS/R to support your plans?		\|'''What is your model for extreme-scale I/O, and how do you expect the OS/R to support your plans?'''
	\|''OpenX supports a multiplicity of I/O support interfaces from conventional file systems to a unified namespace and asynchronous control (under separate funding).''		\|OpenX supports a multiplicity of I/O support interfaces from conventional file systems to a unified namespace and asynchronous control (under separate funding).
	\|~~(TG)~~		\|Proxy call to external hosts.
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
Line 93:		Line 93:
	\|For PIPER this refers to performance information: hierarchical aggregation through MRNet to reduce data and perform online analysis. OS support required for bootstrapping / optionally, could be integrated with OS backplanes		\|For PIPER this refers to performance information: hierarchical aggregation through MRNet to reduce data and perform online analysis. OS support required for bootstrapping / optionally, could be integrated with OS backplanes
	\|- style="vertical-align:top;"		\|- style="vertical-align:top;"
	\|Does your project include support for multiple, hierarchical, memory regions? If so, how will it be allocated and managed, by your xstack project or my the OS/R? What APIs do you expect the OS/R to support complex and deep memory?		\|'''Does your project include support for multiple, hierarchical, memory regions? If so, how will it be allocated and managed, by your xstack project or my the OS/R? What APIs do you expect the OS/R to support complex and deep memory?'''
	\|''Physical memory is managed by LXK and allocated to HPX ParalleX processes, which aggregate physical blocks to comprise a hierarchy of logical memory resources. Memory is globally accessible through capabilities based addressing and exported process methods.''		\|Physical memory is managed by LXK and allocated to HPX ParalleX processes, which aggregate physical blocks to comprise a hierarchy of logical memory resources. Memory is globally accessible through capabilities based addressing and exported process methods.
	\|~~(TG)~~		\|Yes, OCR supports multiple, hierarchical memory regions. We need support from the underlying system-software/OS to inform the runtime of the regions that it can use and of their characteristics.
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)

imported>Saman at 21:59, May 9, 2014

2014-05-09T21:59:26Z

← Older revision		Revision as of 21:59, May 9, 2014
Line 57:		Line 57:
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\|(D-TEC)
			- Fault detection notifications from hardware and OS.
			- Fine-grain power measurements (per socket)
			- Thermal information (per socket)
			- Access to DVFS settings

	\|(DynAX)		\|(DynAX)
	\|(X-TUNE)		\|(X-TUNE)

imported>Groved at 21:27, May 7, 2014

2014-05-07T21:27:39Z

← Older revision		Revision as of 21:27, May 7, 2014
Line 44:		Line 44:
	\|(TG)		\|(TG)
	\|(DEGAS)		\|(DEGAS)
	\| Virtualization is an orthogonal issue for D-TEC.		\| Virtualization is an orthogonal issue for D-TEC. We do not assume either the presence or absence of virtualization in our work.
	\|(DynAX)		\|(DynAX)
	\|(X-TUNE)		\|(X-TUNE)

imported>Groved at 21:25, May 7, 2014

2014-05-07T21:25:40Z

← Older revision		Revision as of 21:25, May 7, 2014
Line 44:		Line 44:
	\|(TG)		\|(TG)
	\|(DEGAS)		\|(DEGAS)
	\|(D-TEC)		\| Virtualization is an orthogonal issue for D-TEC.
	\|(DynAX)		\|(DynAX)
	\|(X-TUNE)		\|(X-TUNE)

← Older revision		Revision as of 04:56, May 20, 2014
Line 9:		Line 9:
	* [[#table3\|What energy/power and reliability APIs do you expect from the OS/R?]]		* [[#table3\|What energy/power and reliability APIs do you expect from the OS/R?]]
	* [[#table4\|Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?]]		* [[#table4\|Please describe how parallel programs are "composed" within your project, and what APIs and support is required from the OS/R?]]
	* [[#table5\|]]		* [[#table5\|What is your model for extreme-scale I/O, and how do you expect the OS/R to support your plans?]]
	* [[#table6\|~~After a bug is discovered~~, do you ~~want automated~~ support ~~to create a simplified test that will reproduce the bug~~?]]		* [[#table6\|Does your project include support for multiple, hierarchical, memory regions? If so, how will it be allocated and managed, by your xstack project or my the OS/R? What APIs do you expect the OS/R to support complex and deep memory?]]

	<div id="table1"></div>{{		<div id="table1"></div>{{