{"slug": "do-concurrent-selective-accelerator-host-device-execution", "title": "Do concurrent selective accelerator? host, device execution", "summary": "A Fortran developer proposes adding a RESIDENT locality specifier to DO CONCURRENT to prevent implicit offload copying in systems with separate memory address spaces, allowing selective host-only parallel execution without OpenMP or OpenACC pragmas. The proposal aims to simplify GPU acceleration workflows for NVIDIA, AMD, and Intel compilers by decoupling iteration independence from memory relocation.", "body_md": "I am working with a code where `do concurrent`\n\nis used quite a bit for GPU acceleration. One interesting issue is that not everything needs to be necessarily done on the GPU, certain things would be nice to keep on the host but be able to parallelize on the host. Say initializing some arrays that will also have a host value, or something akin to that.\n\nWith `do concurrent`\n\nthings will get offloaded to the GPU or stay on the CPU but there is now way to tell the runtime/compiler if something is to always be executed on the host/device without using openacc or openmp pragmas. Since we already have `do concurrent`\n\nit might be nice to have something that tells the runtime if we want it to be offloaded to communicate to the compiler backend. This, I hope, will make life easier for nvidia, amd, and intel and anyone that wants to accelerate `do concurrent`\n\n.\n\nI had asked Mr. Claude for some opinions and it put together an interesting proposal. A keyword for `do concurrent`\n\nthat ensures where the data is, something like `do concurrent (..) resident(var1,var2)`\n\nthat tells the compiler that the `DC`\n\nloop will only access data from where it already is. Probably checking some pointer addresses, whatever. The full proposal written by Claude is here, I did not change anything Claude generated because I thought it is interesting to see what it proposed. I iterated a bit with it, talking about pointers, dervied types, etc. but I probably wasn’t exhaustive enough. I think this might be applicable and extendable to any separate memory accelerators, maybe FPGAs!\n\nThis mostly stems from having to do some things on the host (for convenience, or just not worth offloading) but they can be run in parallel. `do concurrent`\n\nis super nice to say that but it has its limitations in systems with separate memory address spaces.\n\nI also thought Claude put together something very nice though. It is quite coherent.\n\nDO CONCURRENT: locality specifier to keep operands co-resident (no implicit offload copy)\n\nMotivation\n\n`DO CONCURRENT`\n\nasserts that loop iterations are independent. On implementations that offload (e.g. nvfortran `-stdpar=gpu`\n\n, ifx `-fopenmp-target-do-concurrent`\n\n), that single assertion is also taken as permission to execute the loop against a *copy* of its operands placed in a separate memory. The offload decision is translation-unit-wide: there is no way, in standard Fortran, to keep one `DO CONCURRENT`\n\nloop parallel-on-host while another is offloaded.\n\nThis conflates two distinct properties:\n\n- the iterations are independent, and\n- the implementation may relocate/duplicate the operands into a distinct memory to run them.\n\nThere is a legitimate need to keep (1) while withdrawing (2) — for example, cheap initialization loops whose arrays are also used on the host, where offloading forces an unwanted migration and is slower than staying put. Today the only workarounds are splitting translation units by `-stdpar`\n\nmode, or dropping `DO CONCURRENT`\n\nfor `!$omp parallel do`\n\n, which leaves standard, single-source Fortran.\n\nProposed solution\n\nAdd an opt-in locality specifier to the `DO CONCURRENT`\n\nheader that asserts the loop executes against the storage its operands already occupy — the implementation may still parallelize the loop, but may not satisfy it by introducing a copy of those operands in a distinct memory.\n\nPlaceholder keyword: `RESIDENT`\n\n(bikeshed later — `COLOCATED`\n\n, `IN_PLACE`\n\n, etc.).\n\n```\ndo concurrent (i = 1:n)  local(tmp)  resident(a, b)\n   tmp  = b(i) * scale\n   a(i) = tmp\nend do\n```\n\nSemantics\n\nThe specifier defines a simple, monotonic set of states:\n\n**Unmarked** — no constraint on operand location. The implementation may run the loop in place, on host threads, or by copying operands into a distinct memory and offloading, whichever it judges best. This is exactly today’s behavior.\n`RESIDENT(...)`\n\n— for each named variable, the loop shall access that variable in the storage it already occupies; the implementation shall not, as a means of executing the loop, create a copy of the variable in physical memory distinct from that storage.\n\nThere is deliberately no third keyword to *force* offload. The permission to offload already exists — it is precisely what an unmarked `DO CONCURRENT`\n\ngrants — so “offload this loop” is just the absence of the constraint together with an implementation that chooses to. `RESIDENT`\n\nonly *withdraws* that permission for the named operands; it introduces no new freedom.\n\nThe actor throughout is the *implementation*, which subsumes both compiler and runtime; whether the placement choice is made at compile time or run time is not constrained.\n\nThe specifier constrains **operand placement only**. Iteration independence and the existing localization rules are unchanged, and it is intended to compose with `DEFAULT(NONE)`\n\n.\n\nNote this is deliberately phrased around *access / no implicit copy*, not around “same address space.” A single-address-space guarantee is too weak: managed/unified-virtual-address memory satisfies it while still physically migrating pages on access — the exact movement this feature exists to prevent. The no-implicit-copy formulation forbids both the discrete-memory copy and the managed-memory migration, and never needs the standard to define “address space.”\n\nThe words *host*, *device*, *GPU*, and *accelerator* do not appear. The desired behavior (parallel, on-host, no migration) arises as a *consequence* on hardware with distinct memories, not as a named target.\n\nIndirection and derived types\n\nFor an entity reached through a descriptor or handle, the constraint applies to the storage that entity *designates* at the point of execution: for a pointer, its target; for an allocatable, its allocated storage. An unassociated pointer or unallocated allocatable designates no storage and the specifier is a no-op for it.\n\nThe constraint is **transitive**. For a derived-type entity, it follows `POINTER`\n\nand `ALLOCATABLE`\n\ncomponents down the reachable graph, so that naming an aggregate pins the storage of its dynamically-sized components as well. This is the same reachability that OpenACC and OpenMP must already traverse for correctness: a *shallow* map of a derived type with allocatable components produces a valid top-level object whose components still designate host storage, which is why both models perform a deep copy — allocating device mirrors for the components and patching the on-device descriptors to point at them (attach/detach). A non-transitive `RESIDENT`\n\nwould reintroduce exactly that shallow-map bug (a pinned aggregate designating data that moved), so transitivity is required for the specifier to be consistent, not merely convenient.\n\nCrucially, `RESIDENT`\n\nis the *prohibition* side of deep copy, which is why the transitive scope carries almost no implementation cost. Deep copy is hard because it must walk the type, allocate mirrors, transfer data, and fix up descriptors. `RESIDENT`\n\ndoes none of that — it only forbids that walk-and-mirror from being performed for the named entity. The scope is the whole reachable graph; the obligation is to leave it alone.\n\nThe one genuine wrinkle is `POINTER`\n\ncomponents: reachability through a pointer is a runtime property (what it is associated with when the loop executes), so the prohibition is well-defined but its *extent* is only known dynamically. `ALLOCATABLE`\n\ncomponents do not have this issue — their storage identity is fixed while allocated.\n\nBehavior across memory architectures\n\n**Distinct physical memories** (discrete GPU): offload requires a copy, which is forbidden; the loop runs with host parallelism. This is the motivating case.\n**Managed / unified virtual memory**: a migration is an implicit copy, so it too is forbidden; the loop stays put.\n**Physically unified/coherent memory**: no distinct memory exists, so the constraint is vacuously satisfied and the implementation is free to run the loop anywhere. This is correct — the reason to pin (avoiding migration) does not exist there.\n**CPU-only**: trivially satisfied; no effect.\n\nThis is the key advantage over an explicit `OFFLOADABLE`\n\n/`NON_OFFLOADABLE`\n\nflag: a device flag is *wrong* on unified memory, whereas a co-residency assertion is automatically right everywhere and its meaning simply evaporates when the hardware makes it moot.\n\nBackwards compatibility\n\nPurely additive and opt-in. Unmarked `DO CONCURRENT`\n\nretains today’s behavior exactly, and users annotate the exceptions rather than the common path.\n\nThe feature adds no new default behavior and changes no existing program. Every conforming implementation already gets to decide operand placement for unmarked loops; this proposal leaves that untouched and only gives the programmer a way to say “not this one.” Because it *withdraws* a permission rather than *granting* a new mode, it cannot alter the meaning of any existing code and is provably zero-cost to adopt.\n\nPrior art\n\n- F2018 locality specifiers (\n`LOCAL`\n\n, `LOCAL_INIT`\n\n, `SHARED`\n\n, `DEFAULT`\n\n) and F2023 `REDUCE`\n\n— the established mechanism for extending the `DO CONCURRENT`\n\nheader with data-semantic clauses.\n- C++ execution policies (\n`seq`\n\n, `par`\n\n, `par_unseq`\n\n): parallelism expressed as permission/policy, with device mapping left to the implementation and never named in the language.\n- OpenACC/OpenMP deep copy (attach/detach): the established transitive-reachability semantics for derived types with allocatable/pointer components.\n`RESIDENT`\n\nis the prohibition side of the same traversal.\n\nOpen questions\n\n**Keyword.** `RESIDENT`\n\nis a placeholder.\n**Pointer-component extent.** Transitivity through `POINTER`\n\ncomponents makes the constraint’s extent a runtime property (the association at execution). The prohibition is well-defined; how, if at all, to characterize the extent statically is open. `ALLOCATABLE`\n\ncomponents are unaffected.\n**Interaction with localization.** Kept strictly orthogonal here; should it be *required* to appear with `DEFAULT(NONE)`\n\nto avoid stacking a placement guarantee on ambiguous data semantics?", "url": "https://wpnews.pro/news/do-concurrent-selective-accelerator-host-device-execution", "canonical_source": "https://fortran-lang.discourse.group/t/do-concurrent-selective-accelerator-host-device-execution/10995#post_4", "published_at": "2026-07-09 01:12:47+00:00", "updated_at": "2026-07-09 01:41:07.469330+00:00", "lang": "en", "topics": ["developer-tools", "ai-infrastructure"], "entities": ["NVIDIA", "AMD", "Intel", "Claude"], "alternates": {"html": "https://wpnews.pro/news/do-concurrent-selective-accelerator-host-device-execution", "markdown": "https://wpnews.pro/news/do-concurrent-selective-accelerator-host-device-execution.md", "text": "https://wpnews.pro/news/do-concurrent-selective-accelerator-host-device-execution.txt", "jsonld": "https://wpnews.pro/news/do-concurrent-selective-accelerator-host-device-execution.jsonld"}}