aidp-fusion-bicc — Fusion BICC bulk extract → Spark

Mirrors the official Oracle AIDP sample at oracle-samples/oracle-aidp-samples → data-engineering/ingestion/Read_Only_Ingestion_Connectors.ipynb, which routes BICC through AIDP's built-in aidataplatform format handler.

When to use

• User wants a bulk extract from Fusion (millions of rows, daily snapshots, full-table loads).
• User mentions: "BICC", "Fusion bulk extract", "BI Cloud Connector", "PVO extract".
• Live REST paging would be too slow (>50k rows or daily refresh).

When NOT to use

• For small/live REST queries → use aidp-fusion-rest.

Prerequisites in the AIDP notebook

1. AIDP-side prep (one-time, by an administrator): an EXTERNAL STORAGE profile registered in the AIDP catalog pointing at the OCI Object Storage bucket BICC writes to. The user references it by name via fusion.external.storage — they don't supply OCI credentials in the notebook.
2. Fusion-side requirement: the Fusion user must have a BICC-administrator role (e.g. BIA_ADMINISTRATOR_DUTY). A regular Fusion REST user (Finance Manager persona) gets a 302 to IDCS instead of BICC JSON — confirmed live.
3. The PVO (Public View Object) name and its source schema (e.g. ERP).
4. Helpers on sys.path.

Auth: HTTP Basic (Recommended path = AIDP aidataplatform format)

Option A — AIDP built-in connector (recommended; matches official sample)

import os
from oracle_ai_data_platform_connectors.rest.fusion import read_bicc_via_aidp_format

df = read_bicc_via_aidp_format(
    spark=spark,
    fusion_service_url=os.environ["FUSION_BICC_BASE_URL"],
    username=os.environ["FUSION_BICC_USER"],            # MUST have BICC privileges
    password=os.environ["FUSION_BICC_PASSWORD"],
    schema=os.environ["FUSION_BICC_SCHEMA"],            # e.g. "ERP"
    datastore=os.environ["FUSION_BICC_PVO"],            # the PVO name
    fusion_external_storage=os.environ["FUSION_BICC_EXTERNAL_STORAGE"],
)
df.show(5)
print("rows:", df.count())

The AIDP format handler does the BICC trigger, polling, manifest read, and OCI Object Storage CSV materialization internally — the user just gets a Spark DataFrame.

Equivalent verbatim invocation (same as the official Oracle sample):

df = (
    spark.read.format("aidataplatform")
        .option("type", "FUSION_BICC")
        .option("fusion.service.url", os.environ["FUSION_BICC_BASE_URL"])
        .option("user.name", os.environ["FUSION_BICC_USER"])
        .option("password", os.environ["FUSION_BICC_PASSWORD"])
        .option("schema", os.environ["FUSION_BICC_SCHEMA"])
        .option("fusion.external.storage", os.environ["FUSION_BICC_EXTERNAL_STORAGE"])
        .option("datastore", os.environ["FUSION_BICC_PVO"])
        .load()
)

Option B — Custom REST trigger + manual Object Storage read (fallback)

Only use this when the AIDP aidataplatform connector isn't available on the cluster, when you need a custom polling cadence, or when you want to inspect the MANIFEST.MF directly. NOT validated against the official sample — endpoint paths and response schema are best-effort and may need adjustment per Fusion version.

import os
from oracle_ai_data_platform_connectors.auth import http_basic_session
from oracle_ai_data_platform_connectors.rest.fusion import (
    trigger_bicc_extract, read_bicc_csv_from_object_storage,
)

session = http_basic_session(
    username=os.environ["FUSION_BICC_USER"],
    password=os.environ["FUSION_BICC_PASSWORD"],
    base_url=os.environ["FUSION_BICC_BASE_URL"],
)
prefix = trigger_bicc_extract(
    session=session,
    base_url=os.environ["FUSION_BICC_BASE_URL"],
    offering=os.environ["FUSION_BICC_OFFERING"],
    poll_interval_seconds=30,
    timeout_seconds=3600,
)
df = read_bicc_csv_from_object_storage(
    spark=spark,
    namespace=os.environ["OCI_NAMESPACE"],
    bucket=os.environ["OCI_BUCKET_BICC"],
    prefix=prefix,
)
print("rows:", df.count())

Gotchas

• BICC privileges — Fusion user must hold a BICC-admin role. Without it, /biacm/api/v[12]/* endpoints 302-redirect to IDCS OAuth (HTTP Basic isn't honored). This is the #1 reason live tests fail. (Live-confirmed against the demo pod with Casey.Brown finance-mgr persona — every BICC endpoint redirected.)
• fusion.external.storage is a catalog-managed name, not a URL. Set it up once via AIDP Catalog UI (or oci aidataplatform CLI), then reference by name. The user never types OCI namespace/bucket in the notebook for Option A.
• schema and datastore — these are BICC concepts: schema = offering schema (ERP, HCM, etc.); datastore = PVO name (e.g. FscmTopModelAM.AnalyticsServiceAM). Get them from the BICC console under "Configure Cloud Extract".
• First extract is slow — BICC builds a full snapshot. Subsequent runs are incremental. Plan for >5 min on the first call.
• Schema inference is slow on big CSVs (Option B path). Option A's connector knows the schema in advance from the BICC metadata.

References

• Helpers: scripts/oracle_ai_data_platform_connectors/rest/fusion.py
• Official Oracle AIDP sample: Read_Only_Ingestion_Connectors.ipynb
• BICC docs: https://docs.oracle.com/en/cloud/saas/applications-common/24a/oafsm/