Why must Large Language Models continuously stack structural memory graphs? Why can we not architect a deep learning paradigm that mirrors biological survivalโone that fluidly streams input perturbations forward while autonomously driving toward internal equilibrium? This repository implements the definitive architectural blueprint for a truly Autograd-free deep learning system. #
Modern deep learning architectures suffer from
Inspired natively by the underlying architectural layout of fluid-mesh-hpc
, this project proposes a novel mathematical-physics-driven neural layer that leverages local grid-point finite difference deviations instead of relying on heavy, macro-level global matrix multiplications.
Static memory via autograd insulation: Freezes operational complexity into a strict static$O(1)$ footprint viajax.lax.stop_gradient
, compressing VRAM allocation down to bare inference-level specifications to minimize hardware load. - Algebraic self-alignment via mathematical physics: Employs fluidic vorticity geometric formulations to enforce automated algebraic weight-tensor realignment governed by physical laws as data streams forward-only through a 1D spatial deviation framework ($U = \text{East} - \text{West}$ ). - Hardware fusion for numerical stability: Integrates a fluidic viscosity brake driven by a micro-dissipation coefficient ($\sigma = 0.00003125$ ). Structurally restructures equations into a unified$(\mathbf{W} \times \gamma) + (\alpha \times \Delta)$ layout inside accelerator ALU register files (where$\gamma$ is the fixed decay factor,$\alpha$ is the learning rate, and$\Delta$ is the curl-inversion displacement), forcing the compiler to dispatch single-clock FMA (Fused Multiply-Add) primitives without pipeline stalls.
Consequently, this system scales down macro VRAM consumption to approximately 1/1000 compared to legacy backprop chains, exploring the structural viability of high-resolution PINN topologies within severely resource-constrained hardware environments.
Branchless spatial finite difference via warp shuffles (Warp-Shed Topology)-
Intra-warp register communication: Deploys register-level shuffle intrinsics (__shfl_up_sync
,__shfl_down_sync
) across active fast-path execution tracks (Lane 1โ30) to eliminate redundant global memory access and efficiently extract 1D spatial deviation scans ($U = \text{East} - \text{West}$ ). -
Boundary latency mitigation: Forces fringe threads (Lane 0, 31) and block boundaries to inherit halo-padding data directly from pre-committed shared memory (__shared__
) blocks, mitigating global VRAM re-load stalls.
Mitigating warp divergence via shared memory masking (Garbage Index Masking)-
Isolated drop-zone integration: Introduces a dedicated static garbage attractor slot (GARBAGE_IDX
) at the terminal boundary of the shared scratchpad layout to neutralize pipeline-stalling warp divergence during edge-condition branch execution. -
Concurrent blind store execution: Forces all 256 threads to dispatch concurrent hardware Store commands simultaneously without individual if-else checks, allowing volatile out-of-bound payloads to safely bleed into the garbage zone while validating conditional instruction (SEL) flattening via low-level hardware MUX selectors (pinn_branchless_select_f32
).
Division-free acceleration and runtime anomaly firewalls-
Constant memory lookup layer: Embeds a 64-element reciprocal lookup table (RECIPROCAL_CELL_LUT
) natively inside high-speed constant memory boundaries to entirely bypass heavy floating-point division (FDIV) pipelines, converting operations into single-clock DSP multiplications. -
Branchless silicon firewall: Activates combinational-logic anomaly detection circuits (pinn_check_hardware_anomaly
) to capture NaN/INF or over-threshold spikes without branching instructions, triggering an immediate hardware flush to0.0f
(CLEAN_BASELINE_VAL) inside the register rail the exact moment a breach occurs.
**Physical address-line zero-copy transport pipeline (Zero-Copy Forwarding)**Eliminating PCIe contention: Leveragespybind11
alongside the global accelerator tensor binding specification__cuda_array_interface__
v3 to structurally enforce a 0ns physical data transport tunnel, dropping host-device (H2D/D2H) replication overhead and PCIe bus bandwidth contention to absolute zero.Instruction cache cold routing: Embeds C++20[[unlikely]]
branch protection gates along the data ingress track, isolating exceptional fault-handling assembly out of the instruction cache's hot path to flatten conditional CPU pipeline stall overheads.
**4-channel independent SoA offset decomposition (Strides = 32 Channel Freezing)**Defensive layout freezing: Restructures the layout at the bare-metal byte offset level into 4 independent channel dictionaries (param_w
,spatial_u
,spatial_v
,adaptive_gain
) to conservatively protect against arbitrary layout manipulation (Transpose/Re-stride) or runtime slicing overheads within the high-level JAX/XLA compiler.Memory bus hardware skipping: Decomposes discrete single-precision floating-point byte offsets directly from the physical base address lineโptr_w (+0)
,ptr_sp_u (+4)
,ptr_sp_v (+8)
, andptr_gain (+12)
โlocking the layout stride vector to exactlysizeof(PinnCell32) = 32
. This forces the accelerator memory bus to skip-jump over residual 16-byte metadata and cache padding segments, streaming only clean floating-point components at peak hardware velocity.
**Python garbage collector asynchronous insulation (Empty Deleter Lifecycle Fence)**Neutralizing runtime jitter: Relinquishes physical hardware asset lifecycle management entirely to the low-level silicon layer, operating a custompy::capsule
lifetime fence equipped with an empty lambda deleter to block asynchronous memory-deallocation interrupts or stop-the-world runtime jitter from the Python Garbage Collector (GC).
**Compile-time static layout verification (Compile-Time Sanity Firewall)**Pre-emptive fault intercept: Explicitly deploys C++20static_assert
directives at the compiler stage to guarantee structural footprints hit exactly 32 bytes and physical memory bus boundaries anchor precisely on 32-byte alignments. This eliminates risks of physical layout packing drift or memory segmentation faults (SegFault
) during high-level in-place transformations.
Cleaving backpropagation paths via autograd insulation-
Immediate tracer interception: Trigger-detonateslax.stop_gradient
insulation shields immediately upon data entering the JAX processing perimeter, radically paralyzing runtime tensor-graph tracing chains designed to accumulate activation cache allocations. -
0ns bitwise cleansing gate: Couples the low-level numerical MUX firewallenforce_algebraic_safety_gate
to execute atomic, 0ns flushes of volatile grid segments leaking overflow spikes ($1.0 \times 10^6$ GLOBAL THRESHOLD) or hardware silicon failure markers ($-99.0$ FAULT SIGNATURE) straight into clean zero reference registers. -
AOT compiler caching gimmick: Mobilizes static system pre-warmup tracks (trigger_system_warmup
) powered by 0MB abstract tracer layout profiles (ShapeDtypeStruct
) to pre-emptively lower and lock the execution graph into accelerator primitive caches, permanently eradicating JIT compilation latency jitter at the boot boundary. - Asymptotic complexity reduction: Restructures overall computational memory complexity from a resolution-dependent quadratic$O(N^2)$ scale down to a strict static$O(1)$ layout, collapsing large-scale distributed training memory footprints down to pure bare inference-level specifications.
Physics-driven algebraic residual cancellation (Cross-Axis Curl Inversion)-
Vorticity cross-vectorization: Bypasses iterative backpropagation chains and heavy gradient-descent convergence paths entirely, instead enforcing fluidic vorticity geometric formulations to cross-vectorize the inverted vertical displacement strands into horizontal autonomous weight-rectification vectors (curl_inverted_u
,curl_inverted_v
) via deterministic algebraic synthesis.
Refactoring mathematical layouts for single-clock FMA acceleration (1-Cycle FMA Path)-
Numerical homeostasis brake: Integrates a fluidic viscosity brake driven by a micro-dissipation coefficient ($\sigma = 0.00003125$ SIGMA DISSIPATION) to stabilize tensor updates and neutralize floating-point divergence within an autograd-free runtime context. -
Single-clock primitive compilation: Mathematically rebuilds update equations into an exact$(\mathbf{W} \times \gamma) + (\alpha \times \Delta)$ pipeline topology (where$\gamma$ is the fixedDECAY_FACTOR
,$\alpha$ is thelearning_rate
, and$\Delta$ is the curl-inversion strands). This minimizes arithmetic pipeline stalls inside accelerator ALU register files, forcing the compiler to output exactly 1-cycle hardware FMA (Fused Multiply-Add) primitive machine codes.
Buffer recycling via in-place VRAM overwriting (Donate-Buffer In-place Overwrite)-
Sovereign buffer locking: Enforces explicit static buffer allocation locking inside the macro-level fused integration kernel (_fused_xla_update_step
) using the@functools.partial(jax.jit, donate_argnums=(0,))
directive. -
Zero-copy memory pass-through: Completely liquidates transient VRAM buffer allocation overheads, ensuring updated metrics directly overwrite historical data in-place onto the raw C++ physical address wires (param_w
).
Passive event-driven tracking with strict zero nominal overhead****Eliminating polling overhead: Rejects resource-intensive active polling loops that drain hardware compute threads during runtime, instead operating a highly efficient asynchronous event loop configured to trigger exclusively upon capturing hardware interrupt flags.Strict zero data-path insulation: Freezes nominal telemetry operations to a primitivehardware_marker_signal == 0.0
early-exit track during 99.9% of healthy physical homeostasis states, enforcing aStrict Zero
performance baseline that completely isolates and shields the active AI streaming data path from framework-induced latency jitter.
**Atomic context shielding against high-volume interrupt bursts (Async Mutex Synchronization)**Conservative fault-burst modeling: Establishes a highly conservative fail-safe posture designed to withstand extreme multi-node cascade anomalies where catastrophic numerical explosions or hardware failure tokens (-99.0f
CATASTROPHIC FAULT) burst concurrently from distributed grid banks.Race condition liquidation: Deploys an explicitasyncio.Lock
hardware-synchronized primitive (infrastructure_atomic_lock
) across the 2D topology map registry (hardware_health_registry
) to atomically arbitrate emergency allocation requests and permanently liquidate memory race conditions among multiple critical-state failure nodes.
Virtual address-line routing redirection and live hardware hot-plugging****Zero-power standby isolation: Constructs an isolated emergency backup node pool (defaultcold_standby_pool_size = 5
) where physical host accelerator rails are kept completely unpowered while pre-locking their raw memory address topologies.Dynamic pointer offset hot-swapping: Triggers an instantaneous, cascading pointer offset substitution inside the Python runtime environment upon capturing a weight-profile corruption interrupt, executing live re-routing matrix updates (active_hardware_backup_routes
) with a true 0ns physical memory reallocation profile.Symmetric telemetry telemetry backhaul: Ingests nominal feedback keys (1.0
SYSTEM RECOVERY KEY) the exact moment the underlying neural core completes autonomous algebraic homeostatic alignment, routing real-time recovery status across the 4 independent SoA channels (param_w
,spatial_u
) back to the Human-Machine Interface (HMI) console.
graph TD
%% ์คํ์ผ ์ ์ (Style Definitions)
classDef inputStyle fill:#1a1a1a,stroke:#333,stroke-width:2px,color:#fff;
classDef layerStyle fill:#2d3748,stroke:#4a5568,stroke-width:1px,color:#fff;
classDef alertStyle fill:#742a2a,stroke:#e53e3e,stroke-width:1px,color:#fff;
classDef outputStyle fill:#1c4ed8,stroke:#3b82f6,stroke-width:2px,color:#fff;
%% ๋
ธ๋ ์ ์ (Node Definitions)
INPUT["๐ฅ INPUT STREAM <br> Real-Time CFD Grid Numerical Telemetry Ingress"]:::inputStyle
subgraph L1 ["1. Bare-Metal CUDA Kernel"]
L1_Core["Spatial Gradient Extraction Layer Core"]
L1_1["Warp Shuffle Intrinsics & Shared Memory Padding<br>(Mitigating Global VRAM Re-load Stalls)"]
L1_2["Constant Memory Reciprocal Lookup Table<br>(RECIPROCAL_CELL_LUT 1-Clock DSP Multiplication)"]
L1_3["Garbage Index Masking & pinn_branchless_select_f32<br>(Warp Divergence Flattening & pinn_check_hardware_anomaly Firewall)"]
end
style L1 fill:#1a202c,stroke:#4a5568,color:#fff
subgraph L15 ["1.5 C++ Interlock Bridge"]
L15_Core["Zero-Copy VRAM Tunneling Layer Core"]
L15_1["pybind11 & __cuda_array_interface__ v3<br>(Physical Address-Line 0ns Forwarding Pipeline)"]
L15_2["4-Channel SoA Byte Offset Decomposition ptr_w(+0) to ptr_gain(+12)<br>(sizeof(PinnCell32)=32 & strides=32 Layout Freezing)"]
L15_3["Empty Deleter Lifecycle Fence & C++20 static_assert / [[unlikely]]<br>(Python GC Insulation & Instruction Cache Cold Routing)"]
end
style L15 fill:#1a202c,stroke:#4a5568,color:#fff
subgraph L2 ["2. Autograd-Insulated JAX Core"]
L2_Core["Algebraic Topological Self-Alignment Layer Core"]
L2_1["lax.stop_gradient Initial Detonation & trigger_system_warmup<br>(enforce_algebraic_safety_gate Cleansing & 0MB Virtual JIT Purging)"]
L2_2["Cross-Axis Curl Inversion curl_inverted_u/v & Fluidic Viscosity Brake<br>(SIGMA_DISSIPATION Scaling Homeostasis Divergence Control)"]
L2_3["Arithmetic Layout Refactoring for Hardware FMA Unit<br>(DECAY_FACTOR Fused Multiply-Add 1-Cycle Integration Track)"]
L2_4["@functools.partial & jax.jit donate_argnums=0 Buffer Locking<br>(param_w Sovereign Address In-place Overwrite Completion)"]
end
style L2 fill:#1a202c,stroke:#4a5568,color:#fff
subgraph L3 ["3. Asynchronous Infrastructure Governance"]
L3_Core["Distributed Node Governance Tower<br>(Strict Zero Baseline Nominal Data-Path Insulation)"]
L3_1["CATASTROPHIC_FAULT Failure Interrupt Ingress<br>(-99.0f Hardware Fault Marker Asynchronous Pinpoint Scanning)"]
L3_2["infrastructure_atomic_lock Mutex Primitive Engagement<br>(hardware_health_registry Node Resource Allocation Race Liquidation)"]
L3_3["Cold Standby Node Pool & active_hardware_backup_routes Matrix<br>(SYSTEM_RECOVERY_KEY Telemetry Backhaul & Address Hot-Swapping)"]
end
style L3 fill:#2d1a2c,stroke:#684a65,color:#fff
OUTPUT["๐ค OUTPUT / HOMEOSTASIS <br> Autograd-Free Real-Time State Topological Alignment & Physical Homeostasis Completion"]:::outputStyle
%% ์ฐ๊ฒฐ์ ์ ์ (Pipeline Datapath Routing)
INPUT --> L1_Core
L1_Core --> L1_1 --> L1_2 --> L1_3
L1_3 --> L15_Core
L15_Core --> L15_1 --> L15_2 --> L15_3
L15_3 --> L2_Core
L2_Core --> L2_1 --> L2_2 --> L2_3 --> L2_4
%% ์ ์ด ๋ฐ ์์ธ ํ๋ฆ (Asynchronous Control & Interrupt Feedback Loop)
L1_3 -. "Hardware Fault Telemetry Monitoring" .-> L3_1
L2_4 -. "Numerical Anomaly Telemetry Monitoring" .-> L3_1
L3_1 --> L3_2 --> L3_3
L2_4 --> OUTPUT
L3_3 -. "Emergency Physical Resource Re-routing" .-> OUTPUT
Tracer graph eradication: Cleaves backpropagation chains immediately upon data entering the JAX processing perimeter, radically liberating VRAM tracking graphs designed to accumulate activation cache profiles. -
JIT latency virtualization: Pairs theenforce_algebraic_safety_gate
ingress firewall with static Ahead-of-Time (AOT) warmup tracks (trigger_system_warmup
) powered by 0MB abstract tracer layout profiles (ShapeDtypeStruct
) to pre-emptively lower and lock the execution graph into accelerator cache lines, permanently neutralizing runtime JIT compilation latency jitter. - Complexity stabilization: Restructures overall computational memory complexity from a resolution-dependent quadratic$O(N^2)$ scale down to a strict static$O(1)$ footprint, compressing large-scale distributed training memory footprints down to pure bare inference-level specifications to minimize framework-induced hardware load.
HBM bottleneck liquidation: Eliminates redundant high-bandwidth memory (HBM) bus probes and instruction latency stalls required to reference neighboring fluidic coordinates during 1D spatial deviation scans ($U = \text{East} - \text{West}$ ). -
Hardware track optimization: Fuses low-level register-interchange intrinsics (__shfl_up_sync
,__shfl_down_sync
) with an isolated garbage attractor address layer (Garbage Index Masking
) at the terminal boundary of the shared scratchpad structure. -
Branchless parallel extraction: Forces 32 concurrent execution strands within a single warp to extract volatile spatial gradient fields at nanosecond intervals through hardware-level MUX selectors (pinn_branchless_select_f32
) completely immune to warp divergence and pipeline-stalling code branches.
Vorticity cross-vectorization: Bypasses iterative backpropagation chains and heavy gradient-descent convergence paths entirely, instead enforcing fluidic vorticity geometric formulations to cross-vectorize the inverted vertical displacement strands into horizontal autonomous weight-rectification vectors (curl_inverted_u
,curl_inverted_v
) via deterministic algebraic synthesis. -
Homeostasis brake integration: Mathematically fuses a fluidic viscosity brake driven by a micro-dissipation coefficient ($\sigma = 0.00003125$ ) to actively stabilize weight updates and neutralize numerical divergence within an autograd-free runtime context. -
Single-clock primitive execution: Restructures update equations into a unified$(\mathbf{W} \times \gamma) + (\alpha \times \Delta)$ topology inside accelerator ALU register files (where$\gamma$ is the fixedDECAY_FACTOR
,$\alpha$ is thelearning_rate
, and$\Delta$ is the curl-inversion displacement), forcing the compiler to output exactly 1-cycle hardware FMA (Fused Multiply-Add) primitive machine codes.
Direct VRAM interlock: Achieves physical-layer tensor binding via the__cuda_array_interface__
v3 specification, mapping only essential operational fields (param_w
,spatial_u
,spatial_v
,adaptive_gain
) from the 32-byte bare-metal layout straight into the JAX compiler view.Bus contention liquidation: Decomposes single-precision floating-point byte offsets directly from the physical base address lineโptr_w (+0)
throughptr_gain (+12)
โcompletely bypassing host-device (H2D/D2H) buffer allocation cycles and physical data replication overheads.Defensive layout freezing: Locks the structural scanning stride to exactly 32 bytes, allowing the accelerator memory bus to skip-jump over residual padding fields to minimize cache-line fragmentation and mitigate hardware bank stalls.
Vertical telemetry integration: Vertically integrates low-level silicon anomaly scanning with macro-level distributed node backup map synthesis to capture immediate hardware failure markers ($-99.0f$ ) at nanosecond thresholds. -
Strict zero data-path insulation: Maintains a passive event-driven control framework that executes a primitivehardware_marker_signal == 0.0
early-exit track during nominal operations, guaranteeing a strict zero compute load that fully insulates the active AI streaming data path. -
Atomic address hot-swapping: Activates the hardware-synchronizedinfrastructure_atomic_lock
Mutex upon capturing an anomaly interrupt to liquidate resource allocation race conditions, executing dynamic pointer offset hot-swapping to unpowered Cold Standby physical node structures with a true 0ns memory reallocation profile.
backend_core.cu
(Layer 1: Bare-Metal CUDA Kernel)-
Finite difference acceleration: Implements 1D spatial finite difference acceleration layouts natively powered by static shared memory padding boundaries and warp shuffle primitives. -
Warp divergence flattening: Houses hardware-level branchless computing loops combining garbage attractor address layers (Garbage Index Masking
) and raw MUX selectors (pinn_branchless_select_f32
) to flatten pipeline conditional jumps. -
Native spec inheritance: Architected to natively inherit and interface with the atomic fault signature tokens and physical layout specifications established by sister infrastructure asset[fluid-mesh-hpc]
v4.
bridge_wrapper.cpp
(Layer 1.5: C++ Interlock Bridge)-
Zero-copy tensor forwarding: Functions as an ultra-fast zero-copy transport channel that directly hooks the__cuda_array_interface__
v3 specification, forwarding VRAM address lines to the JAX compiler view with zero replication costs. -
Defensive layout alignment: Freezes structural footprints to exactlysizeof(PinnCell32) = 32
via stride constraints (strides=32
), decomposing discrete single-precision floating-point byte offsets directly fromptr_w (+0)
throughptr_gain (+12)
. -
Jitter mitigation pipeline: Leverages C++20 static sanity firewalls (static_assert
) and hardware branch modifiers ([[unlikely]]
) to secure strict instruction cache optimization and neutralize runtime memory latency jitter.
pinn_brain.py
(Layer 2: Autograd-Insulated JAX Core)-
Tracer graph eradication: Drives an autograd-free mathematical engine that permanently paralyzes tensor graph accumulation by trigger-detonatinglax.stop_gradient
insulation gates concurrently layer-by-layer. - ** Homeostatic weight realign**: Fuses fluidic viscosity brakes driven by micro-dissipation factors (
$\sigma = 0.00003125$ SIGMA DISSIPATION), 1-cycle FMA compilation paths, and@donate_argnums
in-place memory recycling to enable autonomous weight realignment. -
Infrastructure core interlock: Structurally andๅคง์์ ์ผ๋ก interlocked with the core architectural philosophy and transport transport mechanics of sister infrastructure asset[pim-hbm-bypass]
.
main_orchestrator.py
(Layer 3: Asynchronous Infrastructure Governance)-
Zero-latency data insulation: Operates as a passive event-driven monitoring tower that enforces aStrict Zero
performance baseline during nominal states, completely isolating the active AI streaming datapath from framework overhead. -
Atomic context protection: Deploys the asynchronous primitiveinfrastructure_atomic_lock
Mutex to liquidate resource allocation race conditions during high-volume node failure bursts (-99.0f
CATASTROPHIC FAULT). -
Hot-swapping governance: Governs dynamic pointer offset hot-swapping matrices to mobilize unpowered Cold Standby node slots while symmetrically inheriting the asynchronous homeostatic framework from sister infrastructure asset[fluid-mesh-hpc]
v4.
This project is distributed completely free of charge to the global open-source ecosystem and the mathematical physics academic community under the strict terms of the Apache License 2.0.
Any individual or enterprise is granted full authorization to freely ingest, replicate, modify, distribute, and embed this architecture and source code within commercial hardware or software systems. However, write-ups, commercial deployments, or derivative works must retain explicit copyright attributions and license notification mandates honoring the original author (PJHkorea
).
The forward-only control loop and autonomous tensor realignment systems implemented in this repository constitute a sister architecture systematically integrated at the raw physical address-line level with the author's prior high-end infrastructure assets.
: Shares the definitive blueprint for 0ns physical address-line zero-copy tensor bus direct-coupling via the[pim-hbm-bypass]
(Apache 2.0 Sister Infrastructure)__cuda_array_interface__
v3 specification, alongside the primitive transport mechanics that hijack thelax.stop_gradient
firewall to freeze overall operational complexity into a static$O(1)$ footprint. -
: Natively inherits and interfaces with the evaluation circuit specifications that capture physical pipeline breaches at nanosecond thresholds, trigger-detonating branchless MUX flushes straight to clean zero reference points upon hitting the absolute[fluid-mesh-hpc]
v4 (GNU GPLv3 Sister Infrastructure)$1.0 \times 10^6$ GLOBAL THRESHOLD or capturing the$-99.0$ FAULT SIGNATURE token.
Via this public open-source release, the aforementioned vertically integrated mechanisms automatically secure global legal status as a Defensive Prior Art Registration. While the high-level algorithmic layers presented here (Apache 2.0) are cleared for unrestricted proliferation throughout the ecosystem, any unauthorized expropriation of the underlying silicon-boundary mechanics to pursue monopolistic patent filings within the copyright domain of the sister project (fluid-mesh-hpc
) is legally blocked and barred at the source.
์ LLM์ ๊ธฐ์ต์ ์์๋๊น์? ์๊ทน์ด ์ค๋ฉด ์์ผ๋ก๋ง ํ๋ ค๋ณด๋ด๋ฉฐ, ์ค์ค๋ก ํํ์ ๋ง์ถ๋ ์๋ฌผํ์ ์์กด ๋ฐฉ์์ผ๋ก ๋ง๋ค์ง ๋ชปํ๋ ๊ฑธ๊น์? ์ญ์ ํ(Backprop)๊ฐ ์๋ ๋ฅ๋ฌ๋ ์ฒด๊ณ์ ์ฒญ์ฌ์ง์ ๋ง๋ค์ด๋ดค์ต๋๋ค #
ํ๋ ๋ฅ๋ฌ๋์ ๋ฐฑํ๋กํผ๊ฒ์ด์ (Backpropagation)์ ํตํด ์ฐ์ฐ ๊ทธ๋ํ๊ฐ
๋ณธ ํ๋ก์ ํธ๋ ๊ธฐ์กด ์ ์ ํ๋ก์ ํธ์ธ fluid-mesh-hpc
๊ตฌ์กฐ์์ ์๊ฐ์ ๋ฐ์, ๋ฌด๊ฑฐ์ด ์ ์ญ ํ๋ ฌ ๊ณฑ์ ๋์ ๋ก์ปฌ ๊ฒฉ์์ ์ ์ฐจ๋ถ ํธ์ฐจ๋ฅผ ํ์ฉํ๋ ์๋ฆฌ ๋ฌผ๋ฆฌ ๊ธฐ๋ฐ ์ ๊ฒฝ๋ง ๋ ์ด์ด๋ฅผ ์ ์ํฉ๋๋ค.
์คํ ๊ทธ๋ผ๋ ์ ์ฐ์ ํตํ ์ ์ ๋ฉ๋ชจ๋ฆฌํ:jax.lax.stop_gradient
๋ฅผ ํ์ฉํด ์ฐ์ฐ ๋ณต์ก๋๋ฅผ ์ ์ $O(1)$ ๊ตฌ์กฐ๋ก ๋๊ฒฐํ๊ณ , VRAM ์๋ชจ๋์ ์ถ๋ก (Inference) ์์ค์ผ๋ก ์์ถํ์ฌ ํ๋์จ์ด ๋ถํ๋ฅผ ์ค์ ๋๋ค. - ์๋ฆฌ ๋ฌผ๋ฆฌ ๊ธฐ๋ฐ์ ๋์์ ์์จ ์ ๋ ฌ: ์๋(Vorticity) ๊ธฐํํ ๊ณต์์ ์์ฉํ์ฌ 1์ฐจ์ ๊ณต๊ฐ ํธ์ฐจ($U = \text{East} - \text{West}$ )๋ฅผ ๊ธฐ๋ฐ์ผ๋ก, ๋ฐ์ดํฐ๊ฐ ๋ชจ๋ธ์ ํ ๋ฒ ๊ดํต(Forward-Only)ํ๋ ๋์ ๊ฐ์ค์น ํ ์๊ฐ ๋ฌผ๋ฆฌ ๋ฒ์น์ ๋ฐ๋ผ ๋์์ ์ผ๋ก ์ฌ์ ๋ ฌ๋๋๋ก ํฉ๋๋ค. - ์์น ์์ ์ฑ์ ์ํ ํ๋์จ์ด ์ฐ์ฐ ์ตํฉ: ๋ฏธ์ ์์ฐ ๊ณ์($\sigma = 0.00003125$ ) ๊ธฐ๋ฐ์ ์ ์ฒด ์ ์ฑ ๋ธ๋ ์ดํฌ ํญ์ ์ฌ์ฉํฉ๋๋ค. ๊ฐ์๊ธฐ ALU ๋ด๋ถ ๋ ์ง์คํฐ ๋จ์์ ๊ฐ์ค์น ๊ฐฑ์ ์์์ธ$(\mathbf{W} \times \gamma) + (\alpha \times \Delta)$ ํํ(์ฌ๊ธฐ์$\gamma$ ๋ ๊ณ ์ ๊ฐ์ ์ธ์,$\alpha$ ๋ ํ์ต๋ฅ ,$\Delta$ ๋ ์ปฌ ๋ฐ์ ๋ณ์)๋ก ๋์์ ์ฌ๋ฐฐ์น๋ฅผ ๊ฐํ์ฌ, FMA(Fused Multiply-Add) ์ต์ ํ๋ก ๋ด์์ ๋จ 1์ฌ์ดํด ๋ง์ ํจ์จ์ ์ผ๋ก ์ฒ๋ฆฌ๋๋๋ก ์ ๋ํฉ๋๋ค.
๊ฒฐ๊ณผ์ ์ผ๋ก ํน์ ์๋ฆฌ ๋ฌผ๋ฆฌ ์๋ฎฌ๋ ์ด์ ํ๊ฒฝ์์ VRAM ์๋ชจ๋์ ๊ธฐ์กด ๋๋น ์ฝ 1/1000 ์์ค์ผ๋ก ๋ฎ์ถ์ด, ์ ํ๋ ๋ฆฌ์์ค์์๋ ๊ณ ํด์๋ PINN ์ํคํ ์ฒ๊ฐ ์คํจ์ฑ ์๊ฒ ์๋ํ ๊ฐ๋ฅ์ฑ์ ํ์ํฉ๋๋ค.
์ํ ์
ํ ๊ธฐ๋ฐ์ ๋ฌด๋ถ๊ธฐ ๊ณต๊ฐ ์ฐจ๋ถ (Warp-Shed Topology)- ์ํ ๋ด๋ถ(Lane 1~30)์ ๊ณ ์ ์ฐ์ฐ ๊ตฌ๊ฐ์๋ ๋ ์ง์คํฐ ๊ฐ ์งํต ํต์ ์ธ ์
ํ ์ธํธ๋ฆฐ์ง(
__shfl_up_sync
,__shfl_down_sync
)์ ์ ์ฉํ์ฌ, ์ ์ญ ๋ฉ๋ชจ๋ฆฌ ์ ๊ทผ์ ์ค์ด๊ณ 1์ฐจ์ ๊ณต๊ฐ ํธ์ฐจ($U = \text{East} - \text{West}$ ) ์ค์บ์ ํจ์จ์ ์ผ๋ก ์ ์ถํ๋๋ก ์ ๋ํ์ต๋๋ค. - ์ํ ์ ๋๋จ(Lane 0, 31) ๋ฐ ๋ธ๋ก ๊ฒฝ๊ณ์ ์ค๋ ๋๋ ์ ์ญ ๋ฉ๋ชจ๋ฆฌ ์ฌ์์ฒญ(Re-load) ์ง์ฐ์ ์ํํ๊ณ ์, ์ด๋ฏธ ๊ฐ๋๋ ๊ณต์ ๋ฉ๋ชจ๋ฆฌ(
__shared__
) ํจ๋ฉ ์์ญ์ ๋ฐ์ดํฐ๋ฅผ ์ฌ์ฌ์ฉํ์ฌ ์์๋ฐ๋ ๊ตฌ์กฐ๋ฅผ ์๋ํ์ต๋๋ค.
- ์ํ ๋ด๋ถ(Lane 1~30)์ ๊ณ ์ ์ฐ์ฐ ๊ตฌ๊ฐ์๋ ๋ ์ง์คํฐ ๊ฐ ์งํต ํต์ ์ธ ์ ํ ์ธํธ๋ฆฐ์ง(
๊ณต์ ๋ฉ๋ชจ๋ฆฌ ๊ฐ์ ๋ง์คํน์ ํตํ ์ํ ๋ถ๊ธฐ ๋ถ์ฐ ์ํ (Garbage Index Masking)- ๊ฒฝ๊ณ ์กฐ๊ฑด ์ฒ๋ฆฌ ์ ํน์ ์ค๋ ๋๋ง ๊ณต์ ๋ฉ๋ชจ๋ฆฌ์ ์ ๊ทผํ ๋ ๋ฐ์ํ๋ ์ํ ๋ถ๊ธฐ ๋ถ์ฐ(Warp Divergence)์ ์ค์ด๊ธฐ ์ํด, ๊ณต์ ๋ฉ๋ชจ๋ฆฌ ๋ ์ด์์ ๋งจ ๋๋จ์ ๊ฒฉ๋ฆฌ ์ฌ๋กฏ์ธ ์ฐ๋ ๊ธฐํต ์ฃผ์(
GARBAGE_IDX
) ์์ญ์ ๊ฐ์ค๋ก ๋์
ํ์ต๋๋ค. - 256๊ฐ ์ค๋ ๋๊ฐ ๊ฐ๋ณ ์กฐ๊ฑด๋ฌธ ๋ถ๊ธฐ ์์ด ์ผ์ ํ ๋์นญ Store ๋ช
๋ น์ ์คํํ๋, ์ ํจํ์ง ์์ ๊ฒฝ๊ณ ์ฐ์ฐ ๊ฒฐ๊ณผ๋ ์ฐ๋ ๊ธฐํต ์ฃผ์๋ก ์์ฐ์ค๋ฝ๊ฒ ํก์ยท์ ์ค๋๋๋ก ์ ๋ํ์ฌ ํ๋์จ์ด ๋ ๋ฒจ์ ๋ฌด๋ถ๊ธฐ ์ ํ์(
pinn_branchless_select_f32
) ๊ตฌ์กฐ๋ฅผ ํตํ ์กฐ๊ฑด๋ถ ์ ํ ๋ช ๋ น์ด(SEL) ํํํ๋ฅผ ์คํ์ ์ผ๋ก ํ์ธํด ๋ณด์์ต๋๋ค.
- ๊ฒฝ๊ณ ์กฐ๊ฑด ์ฒ๋ฆฌ ์ ํน์ ์ค๋ ๋๋ง ๊ณต์ ๋ฉ๋ชจ๋ฆฌ์ ์ ๊ทผํ ๋ ๋ฐ์ํ๋ ์ํ ๋ถ๊ธฐ ๋ถ์ฐ(Warp Divergence)์ ์ค์ด๊ธฐ ์ํด, ๊ณต์ ๋ฉ๋ชจ๋ฆฌ ๋ ์ด์์ ๋งจ ๋๋จ์ ๊ฒฉ๋ฆฌ ์ฌ๋กฏ์ธ ์ฐ๋ ๊ธฐํต ์ฃผ์(
๋๋์
์ฐ์ฐ ๋ฐ ์์ธ ์ฒ๋ฆฌ ๊ฐ์ ๊ฐ๋- ๋ถ๋์์์ ๋๋์
์ฐ์ฐ์ด ๊ฐ์๊ธฐ ํ์ดํ๋ผ์ธ์ ์ฃผ๋ ๋์ ์ค๋ฒํค๋๋ฅผ ์ฐํํ๊ธฐ ์ํด, ํ๋์จ์ด ์ค๋ฆฌ์ฝ Constant ๋ฉ๋ชจ๋ฆฌ ์์ญ์ 64์์ ์๋ฐ์ ์ญ์ ๋ฃฉ์
ํ
์ด๋ธ(
RECIPROCAL_CELL_LUT
)์ ๋ด์ฅํ์ฌ ๋จ์ผ ์ฌ์ดํด DSP ๊ณฑ์
์ฐ์ฐ์ผ๋ก ์ ํ์ ๊พํ์ต๋๋ค. - ์์น ํญ๋ฐ(NaN/INF) ๋ฐ ๊ฒฐํจ ๋ง์ปค ์ ์
์, ์ ์ด ํ์ดํ๋ผ์ธ์ ์ ์ฒด๋ฅผ ๋ฐฉ์งํ๊ธฐ ์ํด ๋ถ๊ธฐ๋ฌธ ์๋ ์กฐํฉ ๋
ผ๋ฆฌ ์กฐ๊ฑด์(
pinn_check_hardware_anomaly
)์ ๊ฐ๋ํ์ฌ ์์ธ ์์น ๊ฒ์ถ ์ฆ์ ์ฒญ์ ๋ฒ ์ด์ค๋ผ์ธ(0.0f
) ์ํ๋ก ๋ฆฌ์ ํ ํ๋ถ ์ค๋ฆฌ์ฝ ๋ฐฉํ๋ฒฝ ๋จ์์ ์ฆ๊ฐ ํ๋ฌ์๋๋๋ก ๊ตฌํํ์ต๋๋ค.
- ๋ถ๋์์์ ๋๋์ ์ฐ์ฐ์ด ๊ฐ์๊ธฐ ํ์ดํ๋ผ์ธ์ ์ฃผ๋ ๋์ ์ค๋ฒํค๋๋ฅผ ์ฐํํ๊ธฐ ์ํด, ํ๋์จ์ด ์ค๋ฆฌ์ฝ Constant ๋ฉ๋ชจ๋ฆฌ ์์ญ์ 64์์ ์๋ฐ์ ์ญ์ ๋ฃฉ์ ํ ์ด๋ธ(
๋ฌผ๋ฆฌ ์ฃผ์์ ๊ธฐ๋ฐ์ ์ ๋ก์นดํผ ์์ก ํ์ดํ๋ผ์ธ (Zero-Copy Forwarding)pybind11
๋ฐ ๊ธ๋ก๋ฒ ๊ฐ์๊ธฐ ํ
์ ๋ฐ์ธ๋ฉ ํ์ค ๊ท๊ฒฉ์ธ__cuda_array_interface__
v3๋ฅผ ํ์ฉํ์ฌ, ํธ์คํธ-๋๋ฐ์ด์ค(H2D/D2H) ๊ฐ์ ๋ฌผ๋ฆฌ์ ๋ฐ์ดํฐ ๋ณต์ฌ ์ค๋ฒํค๋์ PCIe ๋์ญํญ ์ ์ ์จ์ ์ ๋ก(0) ์์ค์ผ๋ก ๋ฎ์ถ๋ ๊ฒฝ๋ก๋ฅผ ํ์ํ์ต๋๋ค.- ๋ฐ์ดํฐ ์ธ์
๊ฒฝ๋ก ์์ C++20
[[unlikely]]
๊ฒฝ๊ณ ๋ณดํธ ๊ฒ์ดํธ๋ฅผ ์๋ฒ ๋ฉํ์ฌ, ์์ธ ์ฒ๋ฆฌ ์ด์ ๋ธ๋ฆฌ ์ฝ๋๋ฅผ ๋ช ๋ น์ด ์บ์์ ํซ ํจ์ค ๋ฐ๊นฅ์ผ๋ก ๊ฒฉ๋ฆฌํจ์ผ๋ก์จ CPU ํ์ดํ๋ผ์ธ ์คํจ ์ค๋ฒํค๋๋ฅผ ํํํํ๊ณ ์ ํ์ต๋๋ค.
4์ฑ๋ ๋
๋ฆฝ SoA ์คํ์
๋ถํด ๋ฐ ๋ณดํญ ์ ์ (Strides = 32 Channel Freezing)- ์์ JAX/XLA ์ปดํ์ผ๋ฌ ๋จ์ ์์์ ์ธ ๋ ์ด์์ ๋ณํ(Transpose/Re-stride) ๋ฐ ์ฌ๋ผ์ด์ฑ ์ค๋ฒํค๋๋ฅผ ๋ณด์์ ์ผ๋ก ๋ฐฉ์ดํ๊ณ ์, ํ๋ถ ๋ฌผ๋ฆฌ ๋ฐ์ดํธ ์คํ์
๋ ๋ฒจ์์ 4๊ฐ์ ๋
๋ฆฝ๋ ์ฑ๋ ๋์
๋๋ฆฌ(
param_w
,spatial_u
,spatial_v
,adaptive_gain
)๋ก ๊ตฌ์กฐ๋ฅผ ๋ถํดํ์ต๋๋ค. - ๊ธฐ์ ์ฃผ์์ ์ผ๋ก๋ถํฐ ๋จ์ ๋ฐ๋ ๋ถ๋์์์ ํ๋๋ค์ ๋ฐ์ดํธ ์คํ์
๊ฐ์ฐ ๋ผ์ธ์ธ
ptr_w (+0)
,ptr_sp_u (+4)
,ptr_sp_v (+8)
,ptr_gain (+12)
์ ๊ฐ๋ณ ๋ถํดํ๊ณ , ๋ค์ ์์ ์ค์บ ์คํ์
๋ณดํญ(Strides)์ ๊ตฌ์กฐ์ฒด ์ ์ฒด ํฌ๊ธฐ์ธsizeof(PinnCell32) = 32
๋ก ๊ณ ์ ๊ฒฐ์ฐฉ์์ผ ๊ฐ์๊ธฐ ๋ฉ๋ชจ๋ฆฌ ๋ฒ์ค๊ฐ 16๋ฐ์ดํธ ์ ์ด ๋ฐ ํจ๋ฉ ์์ญ์ ๋ฌผ๋ฆฌ์ ์ผ๋ก ์คํต ์ ํํ๋ฉฐ ํจ์จ์ ์ผ๋ก float ์ฑ๋ถ๋ง ์ฐธ์กฐํ ์ ์๋๋ก ์ ๋ํ์ต๋๋ค.
- ์์ JAX/XLA ์ปดํ์ผ๋ฌ ๋จ์ ์์์ ์ธ ๋ ์ด์์ ๋ณํ(Transpose/Re-stride) ๋ฐ ์ฌ๋ผ์ด์ฑ ์ค๋ฒํค๋๋ฅผ ๋ณด์์ ์ผ๋ก ๋ฐฉ์ดํ๊ณ ์, ํ๋ถ ๋ฌผ๋ฆฌ ๋ฐ์ดํธ ์คํ์ ๋ ๋ฒจ์์ 4๊ฐ์ ๋ ๋ฆฝ๋ ์ฑ๋ ๋์ ๋๋ฆฌ( ํ์ด์ฌ ๊ฐ๋น์ง ์ปฌ๋ ํฐ ๊ฐ์ญ ์ ์ฐ ๊ฐ๋ (Empty Deleter Lifecycle Fence)- ๋ฌผ๋ฆฌ ํ๋์จ์ด ์์์ ๋ฉ๋ชจ๋ฆฌ ์๋ช ์ฃผ๊ธฐ๋ฅผ ํ๋ถ ๋ก์ฐ๋ ๋ฒจ ์์ญ์ ์ผ์ํ๊ณ , ํ์ด์ฌ ๊ฐ๋น์ง ์ปฌ๋ ํฐ(GC)์ ๋น๋๊ธฐ์ ์๊ฑฐ ์๋๋ก ์ธํ ๋ฏธ์ธํ ๋ฐํ์ ์งํฐ(Stop-the-world) ์ง์ ์ ๋ฐฉ์งํ๊ณ ์ ๋น ๋๋ฆฌํฐ(Empty Deleter) ๋๋ค๊ฐ ํฌํจ๋ ์ปค์คํ ์บก์ ํ์ค๋ฅผ ์ ์ฉํด ๋ณด์์ต๋๋ค.
์ปดํ์ผ ํ์ ์ ์ ์ฌ์ ๊ฒ์ฆ ๊ตฌ์กฐ (Compile-Time Sanity Firewall)- C++20 ํ์ค
static_assert
๋ช
์ธ๋ฅผ ๋ช
์์ ์ผ๋ก ๋์
ํ์ฌPinnCell32
๊ตฌ์กฐ์ฒด์ ํฌ๊ธฐ๊ฐ ์ ํํ 32๋ฐ์ดํธ๋ฅผ ๋ง์กฑํ๋์ง, ์ ๋ ฌ(Alignment) ๊ท๊ฒฉ์ด ์ด๊ธ๋์ง ์์๋์ง ๋น๋ ๋จ๊ณ์์ ์๊ฒฉํ ๊ฒ์ฆํ๋๋ก ์ ๋ํ์ต๋๋ค. - ์ด๋ฅผ ํตํด ์์ ๊ฐ์ ํ๋ ์์ํฌ๊ฐ ์ธํ๋ ์ด์ค(In-place) ์กฐ์์ ๊ฐํ ๋ ์ผ์ด๋ ์ ์๋ ๋ฌผ๋ฆฌ ๋ ์ด์์ ๋คํ๋ฆผ ๋ฐ ์ธ๊ทธ๋ฉํ ์ด์ ํดํธ(SegFault) ์ํ์ฑ์ ์ฌ์ ์ ๋ฐฉ์ดํ๊ณ ์ ๋ ธ๋ ฅํ์ต๋๋ค.
-
C++20 ํ์ค
๊ทธ๋ ๋์ธํธ ๊ทธ๋ํ ์์ฑ์ ์ ํํ๋ ์ญ์ ํ ์ฐจ๋จ ๊ฒฉ๋ฆฌ๋ง (Autograd Insulation)- ๋ฐ์ดํฐ๊ฐ JAX ์์ง ์ด์
์ ์ง์
ํ๋ ์ฆ์
lax.stop_gradient
๊ฒฉ๋ฆฌ๋ง์ ์ ์ ์ธ๊ฐํ์ฌ, ์ค๊ฐ ํ์ฑํ ํ
์(Activation) ๋ณด์กด์ ์ํ ์ฐ์ฐ ๊ทธ๋ํ ์ถ์ ์ฌ์ฌ์ ์ฐจ๋จํ๊ณ ์ ํ์ต๋๋ค. - ํ๋ถ ์์น ์ ํ MUX ๊ฒ์ดํธ์ธ
enforce_algebraic_safety_gate
๋ฅผ ๊ฒฐํฉํ์ฌ ์ ๋ ์๊ณ์น์ธ$1.0 \times 10^6$ (GLOBAL THRESHOLD) ๋ฐ ๊ฒฐํจ ํ ํฐ์ธ$-99.0$ (FAULT SIGNATURE) ์ ์
์ขํ๋ฅผ 0ns ๋จ์๋ก ์์์ ํ๋ฌ์ํ๋ ๋ฐฉ์ด์ ์ ๊ตฌ์ถํ์ต๋๋ค. - 0MB ๊ฐ์ ์ถ์ ํ
์ ๋ทฐ(
ShapeDtypeStruct
)๋ฅผ ํ์ฉํ ์์คํ
์ ์ ์์ด ์ปค๋(trigger_system_warmup
)์ ๊ฐ๋ํ์ฌ ์ฒซ ์คํธ๋ฆฌ๋ฐ ์ธ์ ํจ์ค์ JIT ์ปดํ์ผ ๋ ์ดํด์ ์งํฐ๋ฅผ ๋ถํ ์์ ์ ์ ์ ์ ์ผ๋ก ์ ๋กํํ์ต๋๋ค. - ์ด๋ฅผ ํตํด ์ฐ์ฐ ๋ฉ๋ชจ๋ฆฌ ๋ณต์ก๋๋ฅผ ํด์๋ ์ฆ๊ฐ์ ๋ฐ๋ฅธ ์ ๊ณฑ ํํ $O(N^2)$ ์์ ์ ์ $O(1)$ ๋ ์ด์์์ผ๋ก ์ ๋ํจ์ผ๋ก์จ, ๋๊ท๋ชจ ๋ถ์ฐ ํ์ต ์ ํ์ต์ฉ VRAM ์๋ชจ๋์ ๋ํญ ์ ๊ฐํ์ฌ ํ๋์จ์ด ์ธํ๋ผ ๋ถํ๋ฅผ ์ถ๋ก (Inference) ์์ค์ผ๋ก ์์ถํ๋ ์ํคํ ์ฒ๋ฅผ ์๋ํด ๋ณด์์ต๋๋ค.
- ๋ฐ์ดํฐ๊ฐ JAX ์์ง ์ด์ ์ ์ง์ ํ๋ ์ฆ์
๋ฌผ๋ฆฌ ๋ฒ์น ๊ธฐ๋ฐ์ ๋์์ ์์ฐจ ์์ (Cross-Axis Curl Inversion)- ๋ณต์กํ ์ญ์ ํ ๊ทธ๋ ๋์ธํธ ๋์ผํธ ์๋ ด ๊ณผ์ ๋์ , ์ ์ฒด์ ์๋(Vorticity) ๊ธฐํํ ๊ณต์์ ์์ฉํ์ฌ ์์ง ํธ์ฐจ ํญ์ ๋ถํธ๋ฅผ ๋ฐ์ ํ ์ฑ ๊ฐ์ค์น ์์จ ๋ณด์ ๋ณ์ ๋ฒกํฐ(
curl_inverted_u
,curl_inverted_v
)๋ฅผ ์ง์ ๋์ ํฉ์ฑํ๋ ์ต์ ํ ์ฐํ๋ฅผ ๊พํ์ต๋๋ค.
- ๋ณต์กํ ์ญ์ ํ ๊ทธ๋ ๋์ธํธ ๋์ผํธ ์๋ ด ๊ณผ์ ๋์ , ์ ์ฒด์ ์๋(Vorticity) ๊ธฐํํ ๊ณต์์ ์์ฉํ์ฌ ์์ง ํธ์ฐจ ํญ์ ๋ถํธ๋ฅผ ๋ฐ์ ํ ์ฑ ๊ฐ์ค์น ์์จ ๋ณด์ ๋ณ์ ๋ฒกํฐ(
FMA ํ๋์จ์ด ๋ช
๋ น์ด ์ ๋๋ฅผ ์ํ ์์ ์ฌ์ ๊ฐ (1-Cycle FMA Execution Path)- ์คํ ๊ทธ๋ผ๋๊ฐ ๋ฐฐ์ ๋ ํ๊ฒฝ์์ ๊ฐ์ค์น์ ์์น์ ๋ฐ์ฐ์ ์ ์ดํ๊ธฐ ์ํด, ๋ฏธ์ ์์ฐ ๊ณ์์ธ
$\sigma = 0.00003125$ (SIGMA DISSIPATION)๊ฐ ์ฃผ์
๋ ์ ์ฒด ์ ์ฑ ๋ธ๋ ์ดํฌ ํญ์ ์ํ์ ์ผ๋ก ์ ์ฉํ์ต๋๋ค. - ๊ฐ์ค์น ๊ฐฑ์ ์์์
$(\mathbf{W} \times \gamma) + (\alpha \times \Delta)$ ํํ๋ก ์ฌ๋ฐฐ์น(์ฌ๊ธฐ์$\gamma$ ๋ ๊ณ ์ ๊ฐ์ ์ธ์DECAY_FACTOR
,$\alpha$ ๋ ํ์ต๋ฅ learning_rate
,$\Delta$ ๋ ์ปฌ ๋ฐ์ ๋ณ์curl_inverted_u
๋ฐcurl_inverted_v
)ํ์ฌ ๊ฐ์๊ธฐ ALU ๋ด๋ถ ๋ ์ง์คํฐ ๋จ์ ๊ณฑ์ ยท๋ง์ ํ์ดํ๋ผ์ธ ์คํจ์ ์ต์ํํ๊ณ , FMA(Fused Multiply-Add) ์ต์ ํ๋ก ๋ด์์ ๋จ 1์ฌ์ดํด ๋ง์ ํจ์จ์ ์ผ๋ก ํตํฉ ์ฒ๋ฆฌ๋๋๋ก ์กฐ์ฌ์ค๋ฝ๊ฒ ์ ๋ํด ๋ณด์์ต๋๋ค.
- ์คํ ๊ทธ๋ผ๋๊ฐ ๋ฐฐ์ ๋ ํ๊ฒฝ์์ ๊ฐ์ค์น์ ์์น์ ๋ฐ์ฐ์ ์ ์ดํ๊ธฐ ์ํด, ๋ฏธ์ ์์ฐ ๊ณ์์ธ
๋ฒํผ ์ฌ์ฌ์ฉ ๊ธฐ๋ฐ์ ์ธํ๋ ์ด์ค ๊ฐ์ค์น ์ ์ฌ (Donate-Buffer In-place Overwrite)- ์ต์ธ๊ณฝ ์ตํฉ ํ์ดํ๋ผ์ธ(
_fused_xla_update_step
) ๋ฐ ์ ์ ์์ ํจ์ ๊ตฌ์กฐ์@functools.partial(jax.jit, donate_argnums=(0,))
๋ช
์ธ๋ฅผ ๋ฐฐ์นํ์ฌ ๊ฐ์ค์น ๋ฒํผ์ VRAM ์ฌ์ฌ์ฉ์ ์ฒ ์ ํ ๋ฝํนํ์ต๋๋ค. - ์ด๋ฅผ ํตํด ๋งค ์คํ
๋ง๋ค ๋ถํ์ํ ์์ ๋ฒํผ๊ฐ VRAM์ ์ฌํ ๋น๋๋ ๊ธฐํ๋น์ฉ์ ๋ฎ์ถ๊ณ , C++ ๋ฌผ๋ฆฌ ์ฃผ์์ (
param_w
)์ด ๊ฐ๋ฆฌํค๋ ์๋ณธ ๊ฐ์๊ธฐ ๋ฉ๋ชจ๋ฆฌ ์์ญ ์์์๋ง ์์ ์ธํ๋ ์ด์ค(In-place)๋ก ๊ฐ์ค์น๊ฐ ์ง์ ๋ฎ์ด์จ์ง๋๋ก ์ ๋ํ์ต๋๋ค.
- ์ต์ธ๊ณฝ ์ตํฉ ํ์ดํ๋ผ์ธ(
์ด๋ฒคํธ ๊ธฐ๋ฐ์ ์ ๋ก ์ค๋ฒํค๋ ๊ด์ ์ฒด๊ณ (Passive Event-Driven Monitoring)- ํ์์ ์ฐ์ฐ ํ์ฑ ์ํ์์๋ ๋ถํ์ํ ๊ณ์ฐ ์์์ ์๋ชจํ๋ ๋ฌด๊ฑฐ์ด ํด๋ง(Polling) ๋ฃจํ๋ฅผ ๋ฐฐ์ ํ๊ณ , ์ค์ง ํน์ ํ๋์จ์ด ์ธํฐ๋ฝํธ ์ ํธ๊ฐ ์ธ์ ๋ ๋๋ง ๋ฐ์ํ๋ ๋น๋๊ธฐ ์ด๋ฒคํธ ๋ฃจํ ๊ตฌ์กฐ๋ฅผ ์ฑํํ์ต๋๋ค.
- 99.9%์ ์ ์์ ์ธ ๋ฌผ๋ฆฌ ํํ ๊ฐ๋ ์กฐ๊ฑด ํ์์๋ ๊ด์ ๊ณ์ฐ ๋ถํ๋ฅผ ์ต์ํ(
hardware_marker_signal == 0.0
์กฐ๊ฑด ํจ์ค)ํ์ฌ, ๋๊ท๋ชจ AI ๊ฐ์ ์คํธ๋ฆฌ๋ฐ ๊ฒฝ๋ก(Data Path)์ ๋ฏธ์น๋ ๊ฐ์ญ์ ๊ฒฉ๋ฆฌ ์ฐจ๋จํ๋Strict Zero
๋ฒ ์ด์ค๋ผ์ธ์ ์ํํด ๋ณด์์ต๋๋ค.
์์ ๊ฒฝํฉ ๋ฐฉ์ง๋ฅผ ์ํ ๋น๋๊ธฐ ์์์ ๊ฐ๋ (Async Mutex Synchronization)- ํ๋ถ ์ค๋ฆฌ์ฝ ์ปค๋ ๋ฐ ๋ถ์ฐ ๊ฒฉ์์ ๋ฑ
ํฌ์์ ์์น ํญ๋ฐ์ด๋ ํ๋์จ์ด ๊ฒฐํจ ๋ง์ปค์ธ
-99.0f
(CATASTROPHIC FAULT) ์ธํฐ๋ฝํธ๊ฐ ํญ๋ฐ์ ์ผ๋ก ์ ์
(Burst)๋๋ ์ต์
์ ๋ฌผ๋ฆฌ์ ํ๊ณ ์ํฉ์ ๋ณด์์ ์ผ๋ก ์์ ํ์ต๋๋ค. - ๊ณต์ ๋ฐฑ์
์์ ํ์ ์์ ์ฑ์ ํ๋ณดํ๊ธฐ ์ํด
asyncio.Lock
๊ฐ๋ ๋ฉ์ปค๋์ฆ์ธinfrastructure_atomic_lock
์ ๊ฒฐ์ฐฉ์์ผ, 2D ํ ํด๋ก์ง ๋งต ๋ ์ง์คํธ๋ฆฌ(hardware_health_registry
)์ ๋ค์ค ๊ณ ์ฅ ์๋ฆผ ๋ ธ๋ ๊ฐ ์์ ํ ๋น ๊ฒฝ์ ์ํ(Race Condition)๋ฅผ ์์์ ์ผ๋ก ์ ์ดํ๋๋ก ์ ๋ํ์ต๋๋ค.
- ํ๋ถ ์ค๋ฆฌ์ฝ ์ปค๋ ๋ฐ ๋ถ์ฐ ๊ฒฉ์์ ๋ฑ
ํฌ์์ ์์น ํญ๋ฐ์ด๋ ํ๋์จ์ด ๊ฒฐํจ ๋ง์ปค์ธ
๊ฐ์ ์ฃผ์์ ๋ฆฌ๋ค์ด๋ ์
๋ฐ ํซํ๋ฌ๊น
(Cold Standby Address Hot-Swapping)- ์์ ์ ๋ ฅ ์๋ชจ๋ฅผ ์ฐจ๋จํ ์ฑ ๋ฌผ๋ฆฌ ์ฃผ์์ ๋ง ๋ฝํนํด ๋
Cold Standby
์๋น ๊ฐ์๊ธฐ ๋
ธ๋ ํ ํด๋ก์ง ๋งต(๊ธฐ๋ณธcold_standby_pool_size = 5
)์ ์ค๊ณํ์ฌ ๋น์ ์ธํ๋ผ ํ์ ํ๋ณดํ์ต๋๋ค. - ๊ฐ์ค์น ํ๋กํ์ผ ํผ์ ์ธํฐ๋ฝํธ ์ ์ถ ์ฆ์ ํ์ด์ฌ ๋จ์ ํฌ์ธํฐ ์คํ์
๊ต์ฒด(Hot-swap)์ ํจ๊ป ๋ผ์ด๋ธ ๋น์ ๋ผ์ฐํ
๋งต(
active_hardware_backup_routes
)์ ๊ฐฑ์ ํ๋ ์ฐ์ ์ ๋๋ฅผ ๊ฐํํ์ต๋๋ค. - ํํธ ๊ฐ์ค์น๊ฐ ์์จ ๋์ ์ ์ ์ ์๋ฃํ๋ ์๊ฐ ์ ์ฑ ํ๋ฌ๊ทธ ์ ํธ์ธ
1.0
(SYSTEM RECOVERY KEY)์ ์ญ์ผ๋ก ์์
ํ์ฌparam_w(์ค์ฌ ์ ๋์ฅ)
,spatial_u(๋์ ๊ตฌ๋ฐฐ)
๋ฑ 4๋ ๋ ๋ฆฝ ์ฑ๋์ ์ ์ ์์ ๋ณต๊ตฌ ์ฌ๋ถ๋ฅผ ์ธ๊ฐ-๊ธฐ๊ณ ์ธํฐํ์ด์ค(HMI) ๊ด์ ์ ์์ ํ๊ฒ ์ง์กํ๋ ๋ฉ์ปค๋์ฆ ๊ฐ์ด๋๋ผ์ธ์ ์ ๋ฆฝํ์ต๋๋ค.
- ์์ ์ ๋ ฅ ์๋ชจ๋ฅผ ์ฐจ๋จํ ์ฑ ๋ฌผ๋ฆฌ ์ฃผ์์ ๋ง ๋ฝํนํด ๋
graph TD
%% ์คํ์ผ ์ ์
classDef inputStyle fill:#1a1a1a,stroke:#333,stroke-width:2px,color:#fff;
classDef layerStyle fill:#2d3748,stroke:#4a5568,stroke-width:1px,color:#fff;
classDef alertStyle fill:#742a2a,stroke:#e53e3e,stroke-width:1px,color:#fff;
classDef outputStyle fill:#1c4ed8,stroke:#3b82f6,stroke-width:2px,color:#fff;
%% ๋
ธ๋ ์ ์
INPUT["๐ฅ INPUT STREAM <br> ์์นํด์ ๊ฒฉ์์ ๋ฐ์ดํฐ ์คํธ๋ฆผ ์ค์๊ฐ ์ธ์
"]:::inputStyle
subgraph L1 ["1. Bare-Metal CUDA Kernel"]
L1_Core["๊ฒฉ์ ๊ณต๊ฐ ๊ตฌ๋ฐฐ ์ ์ถ ๋ ์ด์ด"]
L1_1["Warp Shuffle Intrinsic ๋ฐ ๊ณต์ ๋ฉ๋ชจ๋ฆฌ ํจ๋ฉ<br>(์ ์ญ ๋ฉ๋ชจ๋ฆฌ ์ฐธ์กฐ ์ง์ฐ ์ํ)"]
L1_2["Constant ๋ฉ๋ชจ๋ฆฌ LUT ๊ธฐ๋ฐ ์ญ์ ์ถ์ถ<br>(RECIPROCAL_CELL_LUT ๋จ์ผ ์ฌ์ดํด ๊ณฑ์
์ ํ)"]
L1_3["Garbage Index Masking ๋ฐ pinn_branchless_select_f32<br>(Warp Divergence ์ํ ๋ฐ pinn_check_hardware_anomaly ๊ฐ๋)"]
end
style L1 fill:#1a202c,stroke:#4a5568,color:#fff
subgraph L15 ["1.5 C++ Interlock Bridge"]
L15_Core["์ ๋ก์นดํผ VRAM ํฐ๋๋ง ๋ ์ด์ด"]
L15_1["pybind11 & __cuda_array_interface__ v3<br>(๋ฌผ๋ฆฌ ์ฃผ์์ ๊ธฐ๋ฐ 0ns ์์ก ํ์ดํ๋ผ์ธ)"]
L15_2["4์ฑ๋ ๋
๋ฆฝ SoA ์คํ์
๋ถํด ptr_w(+0) ~ ptr_gain(+12)<br>(sizeof(PinnCell32)=32 ๋ฐ strides=32 ์ปดํ์ผ๋ฌ ์ต์ ํ)"]
L15_3["Empty Deleter ๋ฐ C++20 static_assert / [[unlikely]] ์์ฑ<br>(ํ์ด์ฌ GC ๊ฐ์ญ ์ ์ฐ ๋ฐ ๋ช
๋ น์ด ์บ์ ํซํจ์ค ๊ฒฉ๋ฆฌ)"]
end
style L15 fill:#1a202c,stroke:#4a5568,color:#fff
subgraph L2 ["2. Autograd-Insulated JAX Core"]
L2_Core["๋์์ ์์ ์์จ ์ ๋ ฌ ๋ ์ด์ด"]
L2_1["lax.stop_gradient ๊ฒฉ๋ฆฌ๋ง ์ด์
๊ธฐํญ / trigger_system_warmup<br>(enforce_algebraic_safety_gate ์์น ์ ํ ๋ฐ 0MB JIT ์งํฐ ์์ ์ ๊ฑฐ)"]
L2_2["๊ต์ฐจ์ถ ์ปฌ ๋ฐ์ curl_inverted_u/v ๋ฐ ์ ์ฑ ๋ธ๋ ์ดํฌ ํญ ๊ฒฐํฉ<br>(SIGMA_DISSIPATION ์ค์ผ์ผ ๋๊ฒฐ ๊ธฐ๋ฐ ์์น ๋ฐ์ฐ ์ ์ด)"]
L2_3["FMA ๋ช
๋ น์ด ์ตํฉ์ ์ํ ์์ ์ฌ์ ๊ฐ Layout<br>(DECAY_FACTOR ๊ณ ์ ๊ฐ์ ์ธ์ ์ฐ๋ 1์ฌ์ดํด ํตํฉ ์ฒ๋ฆฌ)"]
L2_4["@functools.partial ๋ฐ jax.jit donate_argnums=0 ๋ฒํผ ๋ฝํน<br>(param_w ์๋ณธ ๊ธฐ์ฆ ์ฃผ์์ ๊ธฐ๋ฐ ์์ ์ธํ๋ ์ด์ค ์๊ฒฐ)"]
end
style L2 fill:#1a202c,stroke:#4a5568,color:#fff
subgraph L3 ["3. Asynchronous Infrastructure Governance"]
L3_Core["๋ถ์ฐ ๋
ธ๋ ๊ฑฐ๋ฒ๋์ค ์ฌ๋ นํ<br>(hardware_marker_signal ์ ๋ฐ ์ ์ด ๋ฐ ํ์์ ๋ถํ ์ต์ํ)"]
L3_1["CATASTROPHIC_FAULT ๊ฒฐํจ ์ธํฐ๋ฝํธ ์ ํธ ํฌํ<br>(-99.0f ํ๋์จ์ด ํดํธ ๋ง์ปค ๋น๋๊ธฐ ํํฌ์ธํธ ์ค์บ)"]
L3_2["infrastructure_atomic_lock Mutex ๊ฐ๋ ์๋<br>(hardware_health_registry ์์ ํ ๋น ๊ฒฝํฉ ์ํ ์์์ ์ ์ด)"]
L3_3["Cold Standby ์๋น ๋ฌผ๋ฆฌ ๋
ธ๋ ๋ฐ active_hardware_backup_routes<br>(SYSTEM_RECOVERY_KEY ๋๊ธฐํ ๊ธฐ๋ฐ ๋น์ ์ฃผ์์ ํซ์ค์ ์ ์ด)"]
end
style L3 fill:#2d1a2c,stroke:#684a65,color:#fff
OUTPUT["๐ค OUTPUT / HOMEOSTASIS <br> ๋ฏธ๋ถ ์๋ ์ค์๊ฐ ์ํ ์์ ํํ ๋ฐ ๋ฌผ๋ฆฌ ํญ์์ฑ ์๊ฒฐ"]:::outputStyle
%% ์ฐ๊ฒฐ์ ์ ์
INPUT --> L1_Core
L1_Core --> L1_1 --> L1_2 --> L1_3
L1_3 --> L15_Core
L15_Core --> L15_1 --> L15_2 --> L15_3
L15_3 --> L2_Core
L2_Core --> L2_1 --> L2_2 --> L2_3 --> L2_4
%% ์ ์ด ๋ฐ ์์ธ ํ๋ฆ
L1_3 -. "ํ๋์จ์ด ๊ฒฐํจ ๋ชจ๋ํฐ๋ง" .-> L3_1
L2_4 -. "์์น ์์ธ ๋ชจ๋ํฐ๋ง" .-> L3_1
L3_1 --> L3_2 --> L3_3
L2_4 --> OUTPUT
L3_3 -. "์๋น ๋ฌผ๋ฆฌ ์์ ๋ผ์ฐํ
" .-> OUTPUT
์์นํด์ ๋ฐ์ดํฐ๊ฐ ์์ง ์ด์
์ ์ง์
ํจ๊ณผ ๋์์ ๋ฏธ๋ถ ์ฌ์ฌ์ ์ฐจ๋จํ์ฌ, ์ค๊ฐ ํ์ฑํ ํ
์(Activation) ๋ณด์กด์ ์ํ VRAM ์์กด ์ถ์ ๊ทธ๋ํ๋ฅผ ์ฒญ์ฐํ๋๋ก ์ ๋ํ์ต๋๋ค. ์
๊ตฌ ์ ํ ์ฐจ๋จ๋ฌธ์ธ enforce_algebraic_safety_gate
๊ฒ์ดํธ์ 0MB ๊ฐ์ ์ถ์ ํ
์ ๋ทฐ(ShapeDtypeStruct
) ๊ธฐ๋ฐ์ ์ ์ ์์ด ํ์ดํ๋ผ์ธ(trigger_system_warmup
)์ ๊ฒฐํฉํ์ฌ ๋ฐํ์ JIT ์ปดํ์ผ ์งํฐ ๋ ์ดํด์๋ฅผ ์ ์ ์ ์ผ๋ก ๊ฑฐ์ธํ์ต๋๋ค. ์ด๋ฅผ ํตํด ์ฐ์ฐ ๋ฉ๋ชจ๋ฆฌ ๋ณต์ก๋๋ฅผ ๊ณต๊ฐ ํด์๋ ์ฆ๊ฐ์ ๋ฐ๋ฅธ ์ ๊ณฑ ํํ
1์ฐจ์ ๊ณต๊ฐ ์ฐจ๋ถ ํธ์ฐจ(__shfl_up_sync
, __shfl_down_sync
)๊ณผ ์ฃผ์์ ์ ์ด ์ฅ์น์ธ ์ฐ๋ ๊ธฐํต ์ฃผ์ ๋ง์คํน(Garbage Index Masking
) ๊ฐ์ค์ ์ตํฉํ์ฌ, 32๊ฐ ์ค๋ ๋๊ฐ ์ํ ๋ถ๊ธฐ ๋ถ์ฐ(Warp Divergence)์ ๋ฐ๋ฅธ ์คํจ ์์ด ๋ฌด๋ถ๊ธฐ ์ ํ์(pinn_branchless_select_f32
) ํ๋ก๋ฅผ ํตํด ๋๋ ธ์ด ๋จ์๋ก ๊ณต๊ฐ ๊ตฌ๋ฐฐ ๊ฐ๋ฅ์ ๋ณ๋ ฌ ์ ์ถํ๋๋ก ๊ตฌํํด ๋ณด์์ต๋๋ค.
๊ธฐ์กด์ ๊ทธ๋ ๋์ธํธ ๋์ผํธ ํ์์ ์ํํ๋ ๋์ , ์ ์ฒด์ ์๋(Vorticity) ๊ธฐํํ ๊ณต์์ ์์ฉํ์ฌ ์์ง ํธ์ฐจ ํญ์ ๋ถํธ๋ฅผ ๋ฐ์ ํ ์ฑ ๊ฐ์ค์น ์์จ ๋ณด์ ๋ณ์ ๋ฒกํฐ(curl_inverted_u
, curl_inverted_v
)๋ก ๊ต์ฐจ ๋งคํํ๋ ๋ฐฉ์์ ์ทจํ์ต๋๋ค. ์คํ ๊ทธ๋ผ๋๊ฐ ๋ฐฐ์ ๋ ํ๊ฒฝ์์์ ์์น์ ๋ฐ์ฐ์ ์ ์ดํ๊ธฐ ์ํด ๋ฏธ์ ์์ฐ ๊ณ์ DECAY_FACTOR
)์ ํ์ต๋ฅ (learning_rate
)์ด ์ตํฉ๋
CUDA Bare-Metal ๋จ์ 32๋ฐ์ดํธ ๋ฌผ๋ฆฌ ์ ๋ ฌ ๊ตฌ์กฐ์ฒด ๋ ์ด์์์์ ์์ ์ฐ์ฐ์ ํ์์ ์ธ param_w
, spatial_u
, spatial_v
, adaptive_gain
ํ๋๋ง์ __cuda_array_interface__
v3 ํฌ์ธํฐ ์ธํฐ๋ก์ ํตํด JAX ํ
์ ๋ทฐ(View)๋ก ๋ค์ด๋ ํธ ์ฐ๋์ ๊ตฌํํ์ต๋๋ค. ๊ธฐ์ ์ฃผ์์ ์ผ๋ก๋ถํฐ์ ๋ฐ์ดํธ ์คํ์
๊ฐ์ฐ ๋ผ์ธ์ธ ptr_w (+0)
๋ถํฐ ptr_gain (+12)
๊น์ง ๊ฐ๋ณ ๋ถํดํ์ฌ ํธ์คํธ-๋๋ฐ์ด์ค ๊ฐ์ ๋ฌผ๋ฆฌ์ ๋ฒํผ ํ ๋น ๋ฐ ๋ฐ์ดํฐ ๋ณต์ฌ ์ค๋ฒํค๋๋ฅผ ์ฐํํ๊ณ , ๋ค์ ์์ ์ค์บ ์คํ์ ๋ณดํญ์ ๊ตฌ์กฐ์ฒด ์ ์ฒด ํฌ๊ธฐ์ธ 32๋ฐ์ดํธ๋ก ๊ณ ์ ํ์ฌ ๊ฐ์๊ธฐ ๋ฉ๋ชจ๋ฆฌ ๋ฒ์ค ๋ถํ๋ฅผ ์ํํ๊ณ ์บ์๋ผ์ธ ํํธํ ๊ฐ๋ฅ์ฑ์ ์ฌ์ ์ ๋ณด์์ ์ผ๋ก ๋ฐฉ์ดํ๊ณ ์ ํ์ต๋๋ค.
ํ๋ถ ์ค๋ฆฌ์ฝ ๋ ๋ฒจ์์ ์ ์
๋๋ ์์น ํญ๋ฐ ๋ฐ ํ๋์จ์ด ํ์ ์ ํธ์ธ -99.0f
์ค์บ๊ณผ ์์ ๋ถ์ฐ ๋
ธ๋์ ๋ฐฑ์
๋ผ์ฐํ
๋งต ๋น๋๋ฅผ ์์ง์ผ๋ก ์ผ์ฒดํํ๋ ์คํ์ ์ ๊ฐํ์ต๋๋ค. ํ์์์๋ ์ฐ์ฐ ๋ถํ ์ต์ํ(hardware_marker_signal == 0.0
์กฐ๊ฑด ํจ์ค)๋ฅผ ๋ง์กฑํ๋ ํจ์๋ธ ์ด๋ฒคํธ ๊ตฌ๋ํ ์ ์ด ํ๋ ์ธ์ ์ ์งํ๋ค๊ฐ, ๊ฒฐํจ ๋ฐ์ ์ธํฐ๋ฝํธ ํฌํ ์ infrastructure_atomic_lock
Mutex ์๋์ ํตํด ์์ ํ ๋น ๊ฒฝ์ ์ํ(Race Condition)๋ฅผ ์์์ ์ผ๋ก ์ ์ดํ๊ณ 0ns ๋จ์๋ก Cold Standby ์๋น ๋ฌผ๋ฆฌ ๋ ธ๋๋ก ์ฃผ์์ ์ ์ฐํ ์ค์ํํ๋ ๋ฌด์ค๋จ ์์จ ๋ณต๊ตฌ ๊ตฌ์กฐ๋ฅผ ๊ฐ์ด๋๋ผ์ธ์ผ๋ก ์๋ฆฝํ์ต๋๋ค.
backend_core.cu
(Layer 1: Bare-Metal CUDA Kernel)- ๊ณต์ ๋ฉ๋ชจ๋ฆฌ ํจ๋ฉ ์กด ๋ฐ ์ํ ์ ํ ์ธํธ๋ฆฐ์ง ์ฐ๋์ ํตํ 1์ฐจ์ ๊ณต๊ฐ ์ ํ์ฐจ๋ถ ๊ฐ์ ๋ช ์ธ๋ฅผ ๊ตฌํํ ์ปค๋ ์ฝ์ด์ ๋๋ค.
- ์ฐ๋ ๊ธฐํต ์ฃผ์ ๋ง์คํน(
Garbage Index Masking
) ๊ฐ์ค๊ณผ ๋ฌด๋ถ๊ธฐ ์ ํ์(pinn_branchless_select_f32
)๋ฅผ ๊ฒฐํฉํ์ฌ Warp Divergence ๋ถ๊ธฐ๋ฅผ ์ํํ๋ ์ค๋ฆฌ์ฝ ๋จ๋
๊ณ์ฐ ๋ฃจํด์ ํฌํจํ๋ฉฐ, ์๋งค ์ธํ๋ผ ์์ฐ์ธ[fluid-mesh-hpc]
v4์ ๊ฒฐํจ ํ ํฐ ๋ฐ ๋ฌผ๋ฆฌ ๋ ์ด์์ ์คํ์ ๋ค์ดํฐ๋ธ ์์ ๋ฐ ์ฐ๋ํ๋๋ก ์ค๊ณํด ๋ณด์์ต๋๋ค.
bridge_wrapper.cpp
(Layer 1.5: C++ Interlock Bridge)-
__cuda_array_interface__
v3 ๊ท๊ฒฉ์ ์ธํฐ๋กํ์ฌ ๋ฌผ๋ฆฌ ์ฃผ์์ ๊ธฐ๋ฐ์ผ๋ก ๋๋ฐ์ด์ค ๋ฉ๋ชจ๋ฆฌ๋ฅผ JAX๋ก ์ง์กํ๋ ์ ๋ก์นดํผ ์์ก ๊ด๋ก ์คํฌ๋ฆฝํธ์
๋๋ค. - ๊ตฌ์กฐ์ฒด ๋ณดํญ ์ ํ ๊ธฐ๋ฏน(
strides=32
)์ ํ์ฉํดsizeof(PinnCell32) = 32
๊ท๊ฒฉ์ ๋๊ฒฐํ๊ณ ,ptr_w (+0)
๋ถํฐptr_gain (+12)
๊น์ง์ 4์ฑ๋ SoA ๋
๋ฆฝ ํฌ์ธํฐ ์ฃผ์์ ๋ถํด ๋ฐ C++20 ์ ์ ์ฌ์ ๊ฒ์ฆ(static_assert
,[[unlikely]]
) ํ์ดํ๋ผ์ธ์ ํตํด ๋ช ๋ น์ด ์บ์ ์ต์ ํ์ ๋ฉ๋ชจ๋ฆฌ ๋ฐํ์ ์งํฐ ์ ์ด๋ฅผ ์ ๋ํ์ต๋๋ค.
pinn_brain.py
(Layer 2: Autograd-Insulated JAX Core)-
lax.stop_gradient
๊ฒฉ๋ฆฌ๋ง์ ์ด์
๋ฐ ๊ณ์ธต๋ณ๋ก ๊ฒฐ์ฐฉํ์ฌ ํ์ฑํ ํ
์์ VRAM ์์น ๋์ ์ถ์ ๊ทธ๋ํ๋ฅผ ์ต์ํํ๋ ์คํ ๊ทธ๋ผ๋ ํ๋ฆฌ ์๋ฆฌ ์ํ ์์ง์
๋๋ค. - ๋ฏธ์ ์์ฐ ๊ณ์
$\sigma = 0.00003125$ (SIGMA DISSIPATION) ๊ธฐ๋ฐ์ ์ ์ฒด ์ ์ฑ ๋ธ๋ ์ดํฌ ํญ๊ณผ 1์ฌ์ดํด ํ๋์จ์ด FMA ์ฐ์ฐ ์ ๋ ์,@donate_argnums
๊ฐ์ค์น ๋ฒํผ ๊ธฐ์ฆ ๋งค์ปค๋์ฆ์ ์ตํฉํ์ฌ ๊ฐ์ค์น๊ฐ ์ค์ค๋ก ๋์ ์ ๋ ฌ์ ์ด๋ฃฐ ์ ์๋ ๊ฐ๋ฅ์ฑ์ ์คํํ๋ฉฐ, ์์ก ๊ตฌ์กฐ ๋ฐ ์ต์ ํ ๊ฒฝ๋ก ๋จ์์ ์๋งค ์ธํ๋ผ ์์ฐ์ธ[pim-hbm-bypass]
์ ์ค๊ณ ์ฒ ํ๊ณผ ์งํต ๊ฒฐ์ฐฉ๋์ด ์์ต๋๋ค.
main_orchestrator.py
(Layer 3: Asynchronous Infrastructure Governance)- ํ์์ ์ฐ์ฐ ์ค๋ฒํค๋๋ฅผ ์ต์ํ(
Strict Zero
๋ฒ ์ด์ค๋ผ์ธ)ํ์ฌ ๋๊ท๋ชจ AI ๊ฐ์ ๋ฐ์ดํฐ ๊ฒฝ๋ก ๊ฐ์ญ์ ์ฐจ๋จํ๋๋ก ๊ณ ์๋ ํจ์๋ธ ์ด๋ฒคํธ ๊ตฌ๋ํ ์ ์ด ์ฌ๋ นํ์
๋๋ค. - ํ๋ถ ๋ ์ด์ด์์
-99.0f
(CATASTROPHIC FAULT) ๊ฒฐํจ ์ ํธ๊ฐ ๋ค๋ฐ์ ์ผ๋ก ์ธ์
๋ ๋ ์์ ํ ๋น ๊ฒฝ์ ์ํ(Race Condition)๋ฅผ ์ ์ดํ๊ธฐ ์ํinfrastructure_atomic_lock
Mutex ๊ฐ๋ ๋ฐ Cold Standby ๋น์ ์๋น ๋
ธ๋ ์ฃผ์์ ํซ์ค์ ๋งค์ปค๋์ฆ์ ํฌํจํ๋ฉฐ, ์๋งค ์ธํ๋ผ ์์ฐ์ธ[fluid-mesh-hpc]
v4์ ๋น๋๊ธฐ ํญ์์ฑ ์ ์ด ์ฒ ํ์ ์์๋ฐ์์ต๋๋ค.
- ํ์์ ์ฐ์ฐ ์ค๋ฒํค๋๋ฅผ ์ต์ํ(
๋ณธ ํ๋ก์ ํธ๋ Apache License 2.0์ ์๊ฑฐํ์ฌ ์ ์ธ๊ณ ์คํ์์ค ์ํ๊ณ์ ์๋ฆฌ ๋ฌผ๋ฆฌ ํ๊ณ์ ์ ๋ฉด ๋ฌด์ ๋ฐฐํฌ๋ฉ๋๋ค.
๋๊ตฌ๋ ๋ณธ ์ํคํ
์ฒ์ ์์ค์ฝ๋๋ฅผ ์์ ๋กญ๊ฒ ์์
ํ์ฌ ๋ณต์ , ์์ , ๋ฐฐํฌ ๋ฐ ์์ฉ ํ๋์จ์ด/์ํํธ์จ์ด ์ ํ์ ๋ด์ฅํ์ฌ ํ์ฉํ์ค ์ ์์ต๋๋ค. ๋ค๋ง, ์์ฉํ ๋ฐ ํ์ ์ ์๋ฌผ ์์ฑ ์ ์์ ์์(PJHkorea
)์ ์ ์๊ถ ๊ณ ์ง ๋ฐ ๋ผ์ด์ ์ค ์๋ฌด ์ฌํญ์ ๋ช ์ํด ์ฃผ์ ์ผ ํฉ๋๋ค.
๋ณธ ์ ์ฅ์์ ๊ตฌํ๋ ์ ๋ฐฉ ๊ดํต ์ ์ด ๋ฐ ์์จ ๊ฐฑ์ ์์คํ ์ ์ ์์ ์ ํ ํ์ด์๋ ์ธํ๋ผ ์์ฐ๋ค๊ณผ ๋ฌผ๋ฆฌ ์ฃผ์์ ๋ ๋ฒจ์์ ๊ณํต ์ฐ๊ตฌ ๊ฒฐ์ฐฉ๋ ์๋งค ์ํคํ ์ฒ์ ๋๋ค.
:[pim-hbm-bypass]
(Apache 2.0 ์๋งค ์ธํ๋ผ)__cuda_array_interface__
v3 ๊ท๊ฒฉ์ ์ด์ฉํ 0ns ๋ฌผ๋ฆฌ ์ฃผ์์ ์ ๋ก์นดํผ ํ
์ ๋ฒ์ค ์ง๊ฒฐ ๊ตฌ์กฐ ๋ฐlax.stop_gradient
๋ฐฉํ๋ฒฝ์ ์ญ์ด์ฉํ ์ฐ์ฐ ๋ณต์ก๋ ์ ์ $O(1)$ ๋๊ฒฐ ๊ธฐ๋ฏน์ ์์ฒ ์์ก ๊ด๋ก ๊ท๊ฒฉ์ ๊ณต์ ํฉ๋๋ค. -
: ๊ฒฉ์์ ๋ฌผ๋ฆฌ ๊ด๋ก ํ์ด ์ ๋๋
ธ์ด ๋ ๋ฒจ ํ๋์จ์ด ์ฃ์ง ๋จ์์ ๋ฌด๋ถ๊ธฐ MUX ํ๋ก๋ก ์ฆ๊ฐ ํ๋ฌ์ํด ์ฌ๋ฆฌ๋ ์ ๋ ์๊ณ์น[fluid-mesh-hpc]
v4 (GNU GPLv3 ์๋งค ์ธํ๋ผ)$1.0 \times 10^6$ (GLOBAL THRESHOLD) ๋ฐ ๊ฒฐํจ ๋ง์ปค ํ ํฐ$-99.0$ (FAULT SIGNATURE) ํ๊ฐ ํ๋ก ๊ท๊ฒฉ์ ๋ค์ดํฐ๋ธ๋ก ์์ ์ฐ๋ํฉ๋๋ค.
๋ณธ ๊ณต๊ฐ ๋ฐฐํฌ๋ฅผ ํตํด ์ ์์ง ํตํฉ ๋ฉ์ปค๋์ฆ๋ค์ ๊ณต๊ณต์ '๋ฐฉ์ด์ ์ ํ๊ธฐ์ ๋ฑ๋ก(Defensive Prior Art Registration)' ์๊ฒฉ์ ์๋ ํ๋ณดํฉ๋๋ค. ๋ณธ ์์ ์๊ณ ๋ฆฌ์ฆ ๋ ์ด์ด(Apache 2.0)๋ ์ํ๊ณ ์ ๋ฐ์ผ๋ก ์ ํ ์์ด ์ ํ๋๋, ํ๋ถ ์ค๋ฆฌ์ฝ ๊ฒฝ๊ณ๋ฉด์์ ์๋งค ํ๋ก์ ํธ(fluid-mesh-hpc
)์ ์ ์๊ถ ๋๋ฉ์ธ์ ๋ฌด๋จ ์ฌ์ ํํ์ฌ ๋ ์ ์ถ์ํ๋ ค๋ ์๋๋ ๋ฒ์ ์ผ๋ก ์์ฒ ์ฐจ๋จ๋ฉ๋๋ค.