Skip to content

[spark] Expose external table type to Spark#8306

Open
huangxiaopingRD wants to merge 2 commits into
apache:masterfrom
huangxiaopingRD:codex/external-table-type-display
Open

[spark] Expose external table type to Spark#8306
huangxiaopingRD wants to merge 2 commits into
apache:masterfrom
huangxiaopingRD:codex/external-table-type-display

Conversation

@huangxiaopingRD

Copy link
Copy Markdown
Contributor

Purpose

Fix an inconsistency where Spark did not recognize Paimon external tables as external in Spark-facing metadata.
Before this change, a Paimon table could already be stored as table.type=EXTERNAL, but Spark-side logic relies on the reserved property TableCatalog.PROP_EXTERNAL to classify a table as external. Because that property was not exposed from PaimonSparkTableBase, Spark metadata output could report the wrong table type, such as showing a Paimon external table as managed in DESC FORMATTED.
This change fixes that by exposing PROP_EXTERNAL from PaimonSparkTableBase when the underlying Paimon table is external. That keeps Spark behavior consistent with the existing property-based contract already used by Spark-side commands like SHOW TABLE EXTENDED.

Tests

Adds a focused Hive catalog test to verify that:
DESC FORMATTED shows EXTERNAL for external tables
DESC FORMATTED shows MANAGED for managed tables

@huangxiaopingRD huangxiaopingRD force-pushed the codex/external-table-type-display branch from 5858cbe to 582d63b Compare June 20, 2026 16:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant