High-throughput computing

From Wikipedia, the free encyclopedia

High-throughput computing (HTC) is a computer science term to describe the use many computing resources over long periods of time to accomplish a computational task.

1 Challenges
2 High-throughput computing vs. high-performance computing
3 See also
4 References
5 External links

[edit] Challenges

The HTC community is also concerned with robustness and reliability of jobs over a long-time scale. That is, being able to create a reliable system from unreliable components. This research is similar to transaction processing, but at a much larger and distributed scale.

Some HTC systems, such as Condor and PBS, can run tasks on opportunistic resources. It is a difficult problem, however, to operate in this environment. On one-hand the system needs to provide a reliable operating environment for user's jobs, but at the same time the system must not compromise the integrity of the execute node and allow the owner to always have full control of their resources.

[edit] High-throughput computing vs. high-performance computing

There are many differences between high-throughput computing and high-performance computing (HPC). HPC tasks are characterized as needing large of amounts of computing power for short periods of time, whereas HTC tasks also require large amounts of computing, but for much longer times (months and years, rather than hours and days) ^[1]. HPC environments are often measured in terms of FLOPS. The HTC community, however, is not concerned about operations per second, but rather operations per month or per year. Therefore, the HTC field is more interested in how many jobs can be completed over a long period of time instead of how fast an individual job can complete.

As a general rule, HPC systems are tightly coupled parallel jobs, and as such they must execute within a particular site with low-latency interconnects. Conversely, HTC systems are independent, sequential jobs that can be individually scheduled on many different computing resources across multiple administrative boundaries. HTC systems achieve this using various grid computing technologies and techniques.