# Do concurrent selective accelerator? host, device execution > Source: > Published: 2026-07-09 01:12:47+00:00 I am working with a code where `do concurrent` is used quite a bit for GPU acceleration. One interesting issue is that not everything needs to be necessarily done on the GPU, certain things would be nice to keep on the host but be able to parallelize on the host. Say initializing some arrays that will also have a host value, or something akin to that. With `do concurrent` things will get offloaded to the GPU or stay on the CPU but there is now way to tell the runtime/compiler if something is to always be executed on the host/device without using openacc or openmp pragmas. Since we already have `do concurrent` it might be nice to have something that tells the runtime if we want it to be offloaded to communicate to the compiler backend. This, I hope, will make life easier for nvidia, amd, and intel and anyone that wants to accelerate `do concurrent` . I had asked Mr. Claude for some opinions and it put together an interesting proposal. A keyword for `do concurrent` that ensures where the data is, something like `do concurrent (..) resident(var1,var2)` that tells the compiler that the `DC` loop will only access data from where it already is. Probably checking some pointer addresses, whatever. The full proposal written by Claude is here, I did not change anything Claude generated because I thought it is interesting to see what it proposed. I iterated a bit with it, talking about pointers, dervied types, etc. but I probably wasn’t exhaustive enough. I think this might be applicable and extendable to any separate memory accelerators, maybe FPGAs! This mostly stems from having to do some things on the host (for convenience, or just not worth offloading) but they can be run in parallel. `do concurrent` is super nice to say that but it has its limitations in systems with separate memory address spaces. I also thought Claude put together something very nice though. It is quite coherent. DO CONCURRENT: locality specifier to keep operands co-resident (no implicit offload copy) Motivation `DO CONCURRENT` asserts that loop iterations are independent. On implementations that offload (e.g. nvfortran `-stdpar=gpu` , ifx `-fopenmp-target-do-concurrent` ), that single assertion is also taken as permission to execute the loop against a *copy* of its operands placed in a separate memory. The offload decision is translation-unit-wide: there is no way, in standard Fortran, to keep one `DO CONCURRENT` loop parallel-on-host while another is offloaded. This conflates two distinct properties: - the iterations are independent, and - the implementation may relocate/duplicate the operands into a distinct memory to run them. There is a legitimate need to keep (1) while withdrawing (2) — for example, cheap initialization loops whose arrays are also used on the host, where offloading forces an unwanted migration and is slower than staying put. Today the only workarounds are splitting translation units by `-stdpar` mode, or dropping `DO CONCURRENT` for `!$omp parallel do` , which leaves standard, single-source Fortran. Proposed solution Add an opt-in locality specifier to the `DO CONCURRENT` header that asserts the loop executes against the storage its operands already occupy — the implementation may still parallelize the loop, but may not satisfy it by introducing a copy of those operands in a distinct memory. Placeholder keyword: `RESIDENT` (bikeshed later — `COLOCATED` , `IN_PLACE` , etc.). ``` do concurrent (i = 1:n) local(tmp) resident(a, b) tmp = b(i) * scale a(i) = tmp end do ``` Semantics The specifier defines a simple, monotonic set of states: **Unmarked** — no constraint on operand location. The implementation may run the loop in place, on host threads, or by copying operands into a distinct memory and offloading, whichever it judges best. This is exactly today’s behavior. `RESIDENT(...)` — for each named variable, the loop shall access that variable in the storage it already occupies; the implementation shall not, as a means of executing the loop, create a copy of the variable in physical memory distinct from that storage. There is deliberately no third keyword to *force* offload. The permission to offload already exists — it is precisely what an unmarked `DO CONCURRENT` grants — so “offload this loop” is just the absence of the constraint together with an implementation that chooses to. `RESIDENT` only *withdraws* that permission for the named operands; it introduces no new freedom. The actor throughout is the *implementation*, which subsumes both compiler and runtime; whether the placement choice is made at compile time or run time is not constrained. The specifier constrains **operand placement only**. Iteration independence and the existing localization rules are unchanged, and it is intended to compose with `DEFAULT(NONE)` . Note this is deliberately phrased around *access / no implicit copy*, not around “same address space.” A single-address-space guarantee is too weak: managed/unified-virtual-address memory satisfies it while still physically migrating pages on access — the exact movement this feature exists to prevent. The no-implicit-copy formulation forbids both the discrete-memory copy and the managed-memory migration, and never needs the standard to define “address space.” The words *host*, *device*, *GPU*, and *accelerator* do not appear. The desired behavior (parallel, on-host, no migration) arises as a *consequence* on hardware with distinct memories, not as a named target. Indirection and derived types For an entity reached through a descriptor or handle, the constraint applies to the storage that entity *designates* at the point of execution: for a pointer, its target; for an allocatable, its allocated storage. An unassociated pointer or unallocated allocatable designates no storage and the specifier is a no-op for it. The constraint is **transitive**. For a derived-type entity, it follows `POINTER` and `ALLOCATABLE` components down the reachable graph, so that naming an aggregate pins the storage of its dynamically-sized components as well. This is the same reachability that OpenACC and OpenMP must already traverse for correctness: a *shallow* map of a derived type with allocatable components produces a valid top-level object whose components still designate host storage, which is why both models perform a deep copy — allocating device mirrors for the components and patching the on-device descriptors to point at them (attach/detach). A non-transitive `RESIDENT` would reintroduce exactly that shallow-map bug (a pinned aggregate designating data that moved), so transitivity is required for the specifier to be consistent, not merely convenient. Crucially, `RESIDENT` is the *prohibition* side of deep copy, which is why the transitive scope carries almost no implementation cost. Deep copy is hard because it must walk the type, allocate mirrors, transfer data, and fix up descriptors. `RESIDENT` does none of that — it only forbids that walk-and-mirror from being performed for the named entity. The scope is the whole reachable graph; the obligation is to leave it alone. The one genuine wrinkle is `POINTER` components: reachability through a pointer is a runtime property (what it is associated with when the loop executes), so the prohibition is well-defined but its *extent* is only known dynamically. `ALLOCATABLE` components do not have this issue — their storage identity is fixed while allocated. Behavior across memory architectures **Distinct physical memories** (discrete GPU): offload requires a copy, which is forbidden; the loop runs with host parallelism. This is the motivating case. **Managed / unified virtual memory**: a migration is an implicit copy, so it too is forbidden; the loop stays put. **Physically unified/coherent memory**: no distinct memory exists, so the constraint is vacuously satisfied and the implementation is free to run the loop anywhere. This is correct — the reason to pin (avoiding migration) does not exist there. **CPU-only**: trivially satisfied; no effect. This is the key advantage over an explicit `OFFLOADABLE` /`NON_OFFLOADABLE` flag: a device flag is *wrong* on unified memory, whereas a co-residency assertion is automatically right everywhere and its meaning simply evaporates when the hardware makes it moot. Backwards compatibility Purely additive and opt-in. Unmarked `DO CONCURRENT` retains today’s behavior exactly, and users annotate the exceptions rather than the common path. The feature adds no new default behavior and changes no existing program. Every conforming implementation already gets to decide operand placement for unmarked loops; this proposal leaves that untouched and only gives the programmer a way to say “not this one.” Because it *withdraws* a permission rather than *granting* a new mode, it cannot alter the meaning of any existing code and is provably zero-cost to adopt. Prior art - F2018 locality specifiers ( `LOCAL` , `LOCAL_INIT` , `SHARED` , `DEFAULT` ) and F2023 `REDUCE` — the established mechanism for extending the `DO CONCURRENT` header with data-semantic clauses. - C++ execution policies ( `seq` , `par` , `par_unseq` ): parallelism expressed as permission/policy, with device mapping left to the implementation and never named in the language. - OpenACC/OpenMP deep copy (attach/detach): the established transitive-reachability semantics for derived types with allocatable/pointer components. `RESIDENT` is the prohibition side of the same traversal. Open questions **Keyword.** `RESIDENT` is a placeholder. **Pointer-component extent.** Transitivity through `POINTER` components makes the constraint’s extent a runtime property (the association at execution). The prohibition is well-defined; how, if at all, to characterize the extent statically is open. `ALLOCATABLE` components are unaffected. **Interaction with localization.** Kept strictly orthogonal here; should it be *required* to appear with `DEFAULT(NONE)` to avoid stacking a placement guarantee on ambiguous data semantics?