Correctness Tools

QUESTIONS	XPRESS	TG X-Stack	DEGAS	D-TEC	DynAX	X-TUNE	GVR	CORVETTE	SLEEC	PIPER
What kind of runtime overhead can you accept when running your application with a dynamic analysis tools such as a data race detector? 1.1X, 01.5X, 2X, 5X, 10X, 100X.					This should really be a question for the DoE app writers. I imagine that if run in debug mode, performance degrations are acceptable up to a few X. But most likely, if a dynamic tool is modifying program's execution time significantly, it also probably changes the way the program is executed (dynamic task schedules, non-determinism) and hence may analyze irrelevant things.
What types of bugs do you want correctness tools to detect? Data races, deadlocks, non-determinism.					SWARM proposes mechanisms (such as tags) to help avoiding data races. However, SWARM is compatible with traditional synchronization and data access mechanisms, and hence the programmer can create data races. Deadlocks can appear with any programming model based on task graphs, including the one supported by the SWARM runtime. They happen as soon as a cycle is created among tasks. ETI's tracer can help detect deadlocks by showing which tasks were running when the deadlock happened. Non-determinism is often a feature. It may be desired in parts of the programs (for instance when running a parallel reduction), not in others. So a tool for detecting non-determinism per se wouldn't be sufficient, it would require an API to specify when it is unexpected.
What kind of correctness tools can help you with interactive debugging in a debugger? Do you want those tools to discover bugs for you?					Some bugs can be discovered automatically, such as deadlocks and livelocks. For the others, tools need to reduce the time required to find the source of the bug.
Current auto-tuning and performance/precision debugging tools treat the program as a black-box. What kind of white-box program analysis techniques can help in auto-tuning your application with floating-point precision and alternative algorithms?					Compilers can instrument the code to let auto-tuners focus on particular parts of the code and on particular execution characteristics (e.g., by selecting hardware counters). Please define "alternative algorithms".
What kind of testing strategy do you normally apply while developing your software component? Do you write tests before or after the development process? What kind of coverage do you try to achieve? Do you write small unit tests or large systems tests to achieve coverage? Are your tests non-deterministic?					Unit tests are written for each piece of software, and massive system tests are run every day. Tests support some amount of non-determinism, which result in bounded numerical variations in the results. Coverage of the test reflects expected use: often-used components get tested more intensively. System tests are often available before the development process, while unit tests are usually written during and after each code contribution.
After a bug is discovered, do you want automated support to create a simplified test that will reproduce the bug?					This would sound useful. The more simplified, the better.
What is your strategy to debug a DSL and its runtime? What kind of multi-level debugging support do you want for DSLs?					N/A
What kind of visualization and presentation support for bugs do you want from correctness tools? What kind of IDE integration do you want for the correctness tools?					Any tool that allows big-picture and detailed views, as in ETI's tracer for SWARM. Tools that don't integrate with a particular IDE end up being compatible with all IDEs, including vi and emacs, which is very desirable.
When combining various languages/runtimes, can your language/runtime be debugged in isolation from the rest of the system?					The entire execution stack is visible to the programmer at debug time, including the SWARM parts. We do not think that opacity across system parts would bring clarity in the debugging process.
List the testing and debugging challenges that you think the next generation programming languages and models would face?
How can correctness tools help with reasoning about energy? How can correctness tools help with resilience?					Correctness tools may be able to help the programmer make design choices which impact energy consumption, such as enabling non-determinism in parts of their application. Lightweight tools that detect faults are presumably directly applicable to resiliency, for instance by coupling them with checkpointing.