Using Databricks FILE type against data on NAS: what works today with FSx for ONTAP S3 Access Points

A developer tested Databricks' FILE column type against files on Amazon FSx for ONTAP via S3 Access Points, finding that registration works but reads fail. The zero-copy approach is not viable; FILE EXTERNAL and FILE MANAGED variants behave differently, with managed copies succeeding but external references failing. Design tips include using a separate volume for FileSpace and checking ONTAP version via REST API.

A follow-up to Part 2: Databricks and FSx for ONTAP S3 Access Points in the "FSx for ONTAP S3 Access Points x Lakehouse Deep Dive" series. Some results changed since Part 2 — if you read that one, see "Why reads fail" below. I tested whether images and PDFs already sitting on FSx for ONTAP can be referenced from Databricks without copying them into the lakehouse. The result is: registration works, reads do not. If you only need the zero-copy verdict, read "Connecting to an FSx for ONTAP S3 Access Point" and "Design tips" below. If you want the behaviour of the FILE type itself, start from the top. Databricks shipped FILE as a beta column type: a reference to an unstructured file held as a single column in a Delta table. The value is a struct carrying uri / offset / size / content type / checksum . Because it holds the reference and the metadata as a struct, you can pass it straight from a table into an AI function such as ai parse document . There are two variants — FILE EXTERNAL , which holds only the reference, and FILE MANAGED , which copies the bytes into Databricks. I tested this against files on Amazon FSx for NetApp ONTAP, reached through an FSx for ONTAP S3 Access Point — the feature that exposes an ONTAP volume over an S3-compatible API. Here is what I found: FILE EXTERNAL and FILE MANAGED before you ingest GROUP BY and DISTINCT are accepted; = and ORDER BY are rejectedBelow: what is established for each of those four, and how to turn it into design decisions. The templates and scripts are in the repository, so you can re-run the same tests in your own account. Repository: Yoshiki0705/fsxn-lakehouse-integrations https://github.com/Yoshiki0705/fsxn-lakehouse-integrations | Item | Value | |---|---| | Measured | 2026-08-12 | | Storage | One FSx for ONTAP file system | | Access Point | One, INTERNET origin, UNIX root identity | | Workspace | One non-trial workspace, purpose-built, same account and region | | Compute | Serverless SQL warehouse, and a classic DBR 18.2 cluster | | Control | A S3 general bucket in the same account, running the same operations alongside | | Method | SQL Statement Execution API, and boto3 from my workstation | Running the S3 general bucket control alongside matters. Looking only at the Access Point side, you cannot separate a defect in your own environment from behaviour specific to the Access Point. Except where stated otherwise, every conclusion below rests on the same operation succeeding against S3 general bucket. Note that the object tag results depend on the ONTAP tag validation implementation, so a different ONTAP version may behave differently. The FSx console and describe-file-systems do not expose the version; use the ONTAP REST API GET /api/cluster?fields=version to check yours. First, the behaviour of the type itself. There is prior work on this feature from Databricks Japan: パス文字列でもバイナリでもない。DatabricksのFILE型を試す https://qiita.com/taka yayoi/items/ab110946862e63dca9f6 by taka yayoi. Every operational caveat it lists reproduced in my environment, so I have tabulated those first. What follows is my paraphrase and my own measurements, not a translation. Some terms first. FILE MANAGED copies bytes into a Databricks-side location called a FileSpace. list files enumerates files on a Volume and returns FILE values; create file constructs a FILE value explicitly. object metadata reads an object's tags and user metadata. | Behaviour | Measured | Design implication | |---|---|---| | FileSpace on the same volume as the source | CREATE TABLE succeeds; INSERT fails with Cannot get file metadata under managed storage | Give the FileSpace its own volume | checksum on FILE values from list files | Null on every row | Not usable for integrity checks | checksum under FILE MANAGED | Populated with ETAG:"…" , matching the etag from object metadata on S3 general bucket exactly as a string | Comes from the object store, so it is usable for reconciliation | Filenames under FILE MANAGED | Replaced with an opaque UUID; the extension does not survive either | Copy the name to a column before ingesting | Format of uri | Prefixed with dbfs: | Normalise before writing string comparisons | | Serverless notebooks | Unsupported | Use the SQL Statement Execution API against a serverless SQL warehouse | | Automatic garbage collection in beta | Does not run | The FileSpace grows on every re-run. Plan for manual deletion | | The Previews page toggle | Used FILE EXTERNAL / FILE MANAGED without touching it confirmed on 2 workspaces | May not be required. Do not assume either way | content type from create file | binary/octet-stream for both Japanese and ASCII | Pass it explicitly if anything downstream branches on it | What this tells us The difference between FILE EXTERNAL and FILE MANAGED is not only whether bytes are copied. Whether checksum is populated and whether the filename survives both change. Reversing the decision later means re-ingesting, so decide before you ingest. On content type , I got binary/octet-stream for both Japanese and ASCII input. For the same files, list files reported text/plain , so the two paths infer differently. If anything downstream branches on content type , pass it explicitly. Both the documentation and the prior article state that FILE columns cannot be used in grouping expressions. Testing it, support splits more finely than that. | Operation | Result | |---|---| GROUP BY file | accepted | SELECT DISTINCT file | accepted | GROUP BY file.uri | accepted the documented approach | d.file = r.attachment | rejected — The = does not support ordering on type "FILE" | ORDER BY file | rejected — The sortorder does not support ordering on type "FILE EXTERNAL" | What this tells us GROUP BY and DISTINCT both need equality semantics to decide group membership. Both are accepted while the equality operator itself is explicitly rejected. Since the error names the type's operator surface, this does not look like row-count-dependent behaviour. That said, this was confirmed on two rows. I cannot rule out that grouping degrades to identity comparison rather than value equality. In practice, use GROUP BY file.uri and do not depend on GROUP BY file being accepted. This is beta, so the behaviour may change. This is the main question. The answer is "registration works, reads do not", and the cause is not a lack of Access Point support. | Item | Required form | Symptom when wrong | |---|---|---| | External ID in the trust policy | The Databricks account UUID | 403 Forbidden from the storage provider. The metastore ID and the workspace ID do not work | | Self-assume in the trust policy | Account root as the principal, with an aws:PrincipalArn condition naming the role ARN | Naming the role as its own principal fails at creation with Invalid principal , because IAM validates that the principal exists. Unity Catalog requires the role to be able to assume itself | | Resource in the IAM permission policy | The access point ARN arn:aws:s3:<region :<account :accesspoint/<name and .../object/ | AccessDeniedException . The alias-as-bucket-name form alone does not work | | External location URL | The alias form s3://<alias / | An ARN-style URL is rejected immediately with url does not specify a valid bucket name | What this tells us There is an asymmetry worth committing to memory: the same Access Point must be written as an ARN in the IAM policy and as an alias in the external location URL. The AWS CLI works with the alias form of the ARN, so verifying connectivity with the CLI first and then writing the IAM policy will trip you up. The aws:PrincipalArn approach works because that key resolves to the role ARN rather than the assumed-role session ARN. Rather than trust my memory of the condition key, I deployed the template, assumed the role, and assumed it again with those credentials to confirm. With all four satisfied, CREATE EXTERNAL LOCATION succeeds with skip validation=False — meaning Unity Catalog's own validation passed. The external volume on top can be created too. After registration, every read path returns 403 or an authorisation error. The same on serverless SQL and on a classic DBR 18.2 cluster. This is the test that isolated the cause. Unity Catalog will vend the temporary credentials it uses if you ask for them. I requested them for both paths and used them from my workstation, with no Databricks compute or network involved. | Vended for | ListObjectsV2 | HeadObject | |---|---|---| | S3 general bucket control path | success | success | | FSx for ONTAP S3 AP path | AccessDenied | 403 | Same role, same session, same network; the only variable is which path the credentials were scoped to. The error states the cause explicitly: is not authorized to perform: s3:ListBucket on resource: "arn:aws:s3:<region :<account :accesspoint/<name " because no session policy allows the s3:ListBucket action What this tells us because no session policy allows is the decisive part. What is named is the session policy, not the role's permissions. By session policy I mean the down-scoped policy passed to AssumeRole to constrain the role's permissions on a per-request basis. The policy body is generated by Unity Catalog and is not readable by the user. But since adding an access-point-ARN allowance to the role changes nothing, the session policy appears not to mention the access point ARN resource form at all. Only the overlap between the session policy There is no workaround on the user side. Unity Catalog generates the policy, so the fix has to happen there. Emitting the access point ARN form in the session policy should resolve it, though I do not know what implementation constraints apply. Note also that validation at registration time skip validation=False passes while reads return 403. What that validation checks is not published, so I cannot account for the difference. Part 2 of this series measured 2026-05 reported that listing top-level files and reading an explicit file did work through a UC external location. In this run 2026-08-12 , read files , list files , to file and dbutils.fs.ls all returned 403; no read path partially succeeded. That environment is gone, so I cannot account for the difference conclusively. The likely explanation is that the earlier partial successes were observed on an instance-profile path rather than through UC-vended credentials — Part 2 lists instance-profile direct access as a separate approach that works but sits outside UC governance, so the paths may not have been cleanly separated. This run had a S3 general bucket control alongside and includes the vended-credential test outside Databricks entirely, so take these results as the current ones. I will annotate the Part 2 table accordingly. This test has prerequisites. Enable External Data Access on the metastore and grant EXTERNAL USE SCHEMA and EXTERNAL USE LOCATION . Both are disabled by default. They are the controls that allow credentials to be used externally, so they are worth understanding independently of this issue. Arranged so that anyone hitting the same 403 can check top to bottom. None of these is the cause. | What to suspect | Finding | |---|---| | Unity Catalog volume privileges | The failing volume had READ VOLUME ; the working control volume had no privileges at all | | IAM role permissions | Assumed the same role from my workstation and listed successfully. HEAD on a real object returned 200 | HEAD on a missing key returning 403 killing the delta log probe | The Access Point returned 404 for every shape I tried — same as S3 general bucket | x-amz-expected-bucket-owner being rejected | Accepted by both the Access Point and the control bucket | | A different endpoint | Both Databricks and the AWS CLI use <alias .s3.<region .amazonaws.com , which CNAMEs to s3-r-w.<region .amazonaws.com . Same host | | Insufficient compute role permissions | sts:GetCallerIdentity raised NoCredentialsError on the driver. The cluster has no default AWS credentials; the compute role is not in the path | | The S3 gateway endpoint | No change after disassociating the route. The failure reproduces outside the VPC with vended credentials, so this path is not the problem | One note on the last row. A Databricks-managed VPC creates an S3 gateway endpoint at workspace creation, and the private route tables point the S3 managed prefix list at it. That was not the cause here, but for designs that reach an S3 AP from inside a VPC, check first whether the alias resolves into the prefix list's range. I did not reconcile the resolved CIDRs against the prefix list contents in this environment. If you want metadata to live with the file, object tags are one option: tag at write time, read from the table side later. That path has constraints. The following was measured without Databricks in the picture — boto3 from my workstation straight against the FSx for ONTAP S3 Access Point, with the same operations running against the S3 general bucket control bucket. Start with what works. PutObjectTagging , GetObjectTagging and DeleteObjectTagging all function, as do x-amz-meta- headers and x-amz-tagging on the same PutObject as the data. The limits 10 tags per object, 128 characters for a key, 256 for a value are the same values S3 general bucket documents; I did not measure the boundaries. Reading through a second access point on the same volume returned tags written through the first. Tag retention is not scoped per access point. For existing NAS assets shared across several paths, that property works in your favour. But I did not observe where the tags are physically stored, so visibility from NFS or SMB, and retention across SnapMirror, FlexClone and Snapshot restore, remain unverified. Two constraints change how you design. An object overwrite clears tags and user metadata. Nothing errors. A pipeline that rewrites a file has to re-apply them in the same PutObject . My understanding is that S3 general bucket behaves the same way, since PutObject replaces the object, so treat this as a general S3 design point rather than something specific to FSx for ONTAP. I did not run a side-by-side control for this particular case. Tag values are effectively ASCII. Every printable Latin-1 character I tried U+00A1–U+00FF was accepted. Above U+0100, most are rejected with InvalidTag — but not all. A minority of CJK strings are accepted, deterministically, 6 runs out of 6. To find the unit of validation, I tested single characters: | Input | Result | |---|---| | U+5206 | InvalidTag | | U+985E | InvalidTag | | U+5206 U+985E | accepted | | U+6771 | InvalidTag | | U+4EAC | InvalidTag | | U+6771 U+4EAC | InvalidTag | What this tells us Two two-character strings, every constituent character rejected on its own, and the two strings disagree with each other. Validity is determined by the complete byte sequence, not as a function of the characters in it — so this is not a per-character allowlist. For the two-character pairs I tested, results were identical forwards and backwards, and identical as a tag key and as a tag value. I did not test longer strings or other scripts. I raised this with AWS Support. After checking it against the S3 and FSx for ONTAP documentation, they gave the view that the pattern does not match any intentional validation they could identify, and escalated it to the service team as a potential defect in the tag validation layer. Their reply also quoted the documented character set: letters, whitespace, and + - = . : / @ . That does not predict the split under either reading. If "letters" means Unicode letters, every string I tested qualifies yet half are rejected. If it means ASCII letters, all of them should be rejected yet half are accepted. The design guidance is simple: keep tags ASCII and keep localised text in a column. Today the situation is "some strings pass and some do not", which is more awkward to handle than a uniform restriction. I will update the repository and this article when a determination arrives. The same material, ordered the way you actually make the decisions. FILE type FILE EXTERNAL and FILE MANAGED INSERT fails — and because the error surfaces at ingest rather than at configuration time, diagnosis is slower FILE MANAGED . It is replaced with a UUID and the extension does not survive content type inference; pass it explicitly file.uri . Do not depend on GROUP BY file being accepted Unity Catalog and FSx for ONTAP S3 Access Points aws:PrincipalArn condition naming the role ARN .../object/ . The alias-form ARN alone does not work integrations/athena , integrations/glue , integrations/emr-spark , integrations/redshift-spectrum and integrations/lake-formation Object tags PutObject when you rewrite a file. An overwrite clears them How to test All of the above is in the repository as templates and scripts: one template, one script, one runbook. | Artefact | Role | |---|---| integrations/databricks/uc-storage-credential-role.yaml | The IAM role the Unity Catalog storage credential assumes, plus a S3 general bucket control bucket | shared/scripts/probe uc external location.py | Seeds identical object tags on both sides, registers both external locations with validation on, reads object metadata through each, prints a verdict | docs/en/databricks-verification-runbook.md | Prerequisites, where each parameter comes from, the verdict branches, dependency-ordered teardown, measured cost | 1. Baseline first. This is what makes "teardown looks clean" verifiable python3 shared/scripts/audit databricks workspace footprint.py \ --region <region --save /tmp/baseline.json 2. The IAM role and control bucket aws cloudformation deploy --region <region \ --stack-name fsxn-databricks-uc-credential \ --template-file integrations/databricks/uc-storage-credential-role.yaml \ --capabilities CAPABILITY NAMED IAM \ --parameter-overrides file://cfn-params/databricks-uc-storage-credential.json 3. Compare. --control-bucket is deliberately mandatory .venv/bin/python shared/scripts/probe uc external location.py \ --profile <your-profile --role-arn <from stack output \ --ap-alias <alias -ext-s3alias --ap-name <name \ --control-bucket <from stack output --region <region --vend-check 4. Tear down and reconcile against the baseline python3 shared/scripts/audit databricks workspace footprint.py \ --region <region --compare /tmp/baseline.json The script prints one of four verdicts. The first is the important one. One note on the parameter file: a comment key in a CloudFormation parameter file makes the CLI fail with Unknown parameter in Parameters n . The description of each parameter lives in cfn-params/README.md instead, so fill in values with that open alongside. The specific bug is Databricks'. The shape is not, and the shape is the part worth taking away if you are looking at a different product. Managed lakehouse platforms generally reach into your cloud account by assuming a role you create, then constrain that role at runtime with a down-scoped session policy so that credentials handed to a query cannot roam. That is good design. But the policy has to name your storage. If the storage is addressed through an access point while the policy names it in bucket-form ARNs, the two do not match and authorisation fails — however correct your own role policy is. So the question to put to a platform you are evaluating, before you commit to an architecture, can be narrow and answerable: When you vend temporary credentials for an external location, which resource ARN forms does the session policy contain? Specifically, does it include the S3 access point ARN form in addition to the bucket form? And if the platform exposes credential vending, you can verify it without asking anyone: request the credentials, use them from your own machine, and compare a native bucket path against an access point path. That is what --vend-check above does, and it was the only test in this exercise that produced a conclusion with no room for interpretation. I created a test workspace and destroyed it the same day. The workspace mode that uses your own AWS account creates a VPC with a NAT Gateway. A NAT Gateway runs roughly 45 USD a month per gateway hourly charge only, excluding data processing, at the August 2026 Tokyo region rate . These resources are created directly rather than through a CloudFormation stack, so deleting the workspace does not remove them. This is the easiest place to make a foolish mistake during I recorded a baseline before starting and reconciled afterwards. Available NAT Gateways back to zero; VPC, both IAM roles and the bucket gone, with no unassociated Elastic IPs. I turned that check into a script shared/scripts/audit databricks workspace footprint.py . In a shared account a plain listing is not a verdict, because most of what it shows belongs to someone else. Run it once before creating anything and once after teardown; it exits non-zero if anything exists that was not in the baseline. Running it confirmed this teardown was clean and also surfaced leftover roles from an earlier workspace. The cost breakdown is in the runbook with measured figures. Established CREATE STORAGE CREDENTIAL / CREATE EXTERNAL LOCATION / CREATE EXTERNAL VOLUME succeed against an FSx for ONTAP S3 AP alias, with Unity Catalog validation left on object metadata on S3 general bucket returns object tags and user metadata correctly Not established object metadata would read tags through an Access Point if the session policy were fixed ai parse document over files served from an Access Point GROUP BY on a FILE column is correct at scale, or merely accepted Reporting status The object tag behaviour has been escalated through AWS Support to the FSx for ONTAP service team and is awaiting a determination. What I asked for is not a timeline but the distinction: a defect that will be fixed, or an undocumented restriction that gets written down. Either answer lets me state the constraint accurately. The session policy issue and the operator inconsistency go to Databricks when this article publishes. This repository carried an entry from May 2026 saying Unity Catalog external locations do not support S3 Access Points. Registration does work; the entry has been corrected. If you read the earlier version, please note the change. The FILE type is a well-built mechanism for bringing unstructured data under table governance. For referencing files on an FSx for ONTAP S3 Access Point with zero copies, though, it does not get there today. Registration works, so you can try it, but reads fail, and because the cause sits in the session policy on the vending side, users cannot fix it. If zero-copy is the requirement, AWS-native engines that authorise in the caller's own IAM context are the more straightforward path for now — in exchange for assembling table-level governance yourself with Lake Formation. If Unity Catalog governance is the requirement, then for the time being data placement has to be part of the design. The object tag character set is awaiting a determination. When it moves, I will update both the article and the repository. I hope this is useful to someone. See you next time. All test environments have been deleted. The behaviours described are measurements from 2026-08-12 in one specific configuration and may change with platform updates. Beta features are involved, so please check against current documentation before making production decisions.