Sim Theory's Thunder SDK
Simulation Theory has developed and released our patent-pending Thunder Software Development Kit (SDK) to directly address the fundamental challenge of scaling on modern hardware.
Our team of engineers have harnessed their combined experience spanning over a century in high-performance computing, software architecture, hardware architecture, multi-threaded applications, and performance optimization to provide a solution that helps businesses fully utilize the computing resources they already pay for.
The Thunder SDK facilitates the efficient scheduling of work for execution relative to data dependencies. This enables parallel execution without requiring a deep knowledge of multi-threading and complex thread synchronization.
How does the Thunder SDK work?
Sim Theory's Thunder SDK improves application-layer performance to reduce costs, increase throughput, and significantly improve performance on existing hardware.
Optimizes workload execution at the application layer.
Improves performance without requiring major infrastructure changes.
Increases throughput on existing hardware footprints.
Extends the value of CPUs, GPUs, AI accelerators, and edge devices.
Using the SDK, an application can be decomposed into an execution pipeline. For an element in that pipeline, work can be enqueued to execute in parallel. We refer to this as an execution stream. Multiple streams can be enqueued in parallel and submitted to the scheduler for execution.
During the execution of a stream, additional streams can be enqueued and submitted to the scheduler, leading to what we refer to as parallel execution streams. In essence, this enables an application to be pipelined for execution, similar to GPU waves, without the limitation of allowing only one stream, or GPU program, to be able to execute at a time.
How do developers integrate the Thunder SDK?
The Thunder SDK requires integration with application or service source code. It currently provides a C API that facilitates any language that can call C, along with C# bindings for ease of use. Additional bindings are on the development roadmap.
Depending on the complexity of the application or service’s source code, integration can be done at a top-level domain or in a more integrated manner. A typical top-level integration takes anywhere from a few hours to a few days. A deeper, more complex integration generally results in greater performance gains, but the integration time is highly variable and dependent on the complexity of the source code and the performance goals.
Windows, MacOS, and *nix operating systems are supported on x64 and ARM architectures.
What problems can the Thunder SDK solve?
When customers come to Simulation Theory looking to solve a problem or improve an existing solution, we most often find that their processing capability is throttled by at least one of the following five limitations. The Thunder SDK was built to help.
CPU Underutilization - Most software only effectively uses one to four CPU cores, while many modern CPUs contain eight or more cores. The Thunder SDK allows developers to write efficient multi-threaded applications without requiring a deep knowledge of hardware architecture, drivers, or software features.
Irregular Use of the GPU - The GPU cannot receive enough tasks to be fully utilized unless the CPU is being used efficiently enough to process both the work assigned to it and the work necessary to transform data for the GPU. The Thunder SDK makes the CPU efficient enough to send adequate data to the GPU. This efficient use of the GPU means that running cutting edge hardware is not usually necessary.
Memory Access - RAM is slow when compared to CPU speeds. Cloud instances with large amounts of RAM are expensive. When the Thunder SDK is integrated and execution is parallelized with our data-aware scheduler, it is often faster and cheaper to recompute than it is to read cached results from memory.
File IO Speed - Disk access is very slow compared to memory access and CPU speeds. The Thunder SDK enables the mixing of functionally parallel and data parallel execution within the same scheduler, as well as a growing number of asynchronous containers and algorithms to ease use and integration.
Complex or Hidden Performance Challenges - Profiling identifies bottlenecks in performance. Sim Theory can provide general recommendations or custom support.