1 | Extracting Data from Epic with .NET

Authentication

Epic uses OAuth 2.0 Backend Services (SMART on FHIR) together with JWT client assertions to provide secure system-to-system authentication for healthcare integrations.

public async Task<string> GetEpicAccessTokenAsync()
{
    var jwt = BuildSignedJwt(); // RS384, your registered client certificate

    var client = new HttpClient();

    var content = new FormUrlEncodedContent(new[]
    {
        new KeyValuePair<string,string>("grant_type","client_credentials"),
        new KeyValuePair<string,string>(
            "client_assertion_type",
            "urn:ietf:params:oauth:client-assertion-type:jwt-bearer"),
        new KeyValuePair<string,string>("client_assertion", jwt)
    });

    var response = await client.PostAsync($"{epicTokenEndpoint}", content);

    var json = await response.Content.ReadAsStringAsync();

    return JsonDocument.Parse(json)
        .RootElement
        .GetProperty("access_token")
        .GetString();
}

Prefer the Bulk Data API over Individual FHIR Calls

For analytics workloads, Epic’s Bulk Data Export ($export) is the preferred approach. Rather than issuing thousands of Patient, Observation, or Encounter requests, a single asynchronous export job generates NDJSON files that can be processed efficiently.

public async Task<string> KickOffBulkExportAsync(string groupId)
{
    var request = new HttpRequestMessage(
        HttpMethod.Get,
        $"{fhirBaseUrl}/Group/{groupId}/$export");

    request.Headers.Add("Prefer","respond-async");
    request.Headers.Add("Accept","application/fhir+json");

    request.Headers.Authorization =
        new AuthenticationHeaderValue(
            "Bearer",
            await GetEpicAccessTokenAsync());

    var response = await _httpClient.SendAsync(request);

    return response.Headers
        .GetValues("Content-Location")
        .First();
}

Poll for Completion and Download NDJSON Files

Bulk Export is an asynchronous operation. After starting the export, the application periodically polls the status endpoint until Epic returns the completed export manifest.

public async Task<List<string>> PollExportStatusAsync(string statusUrl)
{
    while (true)
    {
        var response = await _httpClient.GetAsync(statusUrl);

        if (response.StatusCode == HttpStatusCode.Accepted)
        {
            await Task.Delay(TimeSpan.FromSeconds(30));
            continue;
        }

        var json = await response.Content.ReadAsStringAsync();

        var manifest =
            JsonSerializer.Deserialize<BulkExportManifest>(json);

        return manifest.Output
            .Select(o => o.Url)
            .ToList();
    }
}

2 | Orchestrating with Azure Data Factory

Azure Data Factory (ADF) does not communicate with FHIR APIs directly. Instead, it orchestrates your .NET extraction service using Azure Function Activities or Web Activities, manages incremental processing, and coordinates data movement throughout the pipeline.

Pipeline Structure

1. Web Activity — Calls your .NET API to initiate the Epic Bulk Export and returns a job identifier.
2. Until Activity — Continuously polls the export status endpoint with configurable wait intervals.
3. Azure Function Activity — Downloads NDJSON files into Azure Data Lake Storage Gen2 using a structured folder hierarchy.
4. Mapping Data Flow / Databricks Notebook — Converts raw FHIR JSON into normalized healthcare entities such as Patient, Encounter, Observation, and Medication.
5. Copy Activity — Loads curated data into Azure SQL Database or Azure Synapse Analytics.
6. Tumbling Window Trigger — Executes incremental processing while maintaining watermarks for every resource type.

Incremental Extraction Pattern

FHIR supports incremental synchronization using the _lastUpdated search parameter, allowing only newly changed resources to be retrieved.

GET /Observation?_lastUpdated=gt2026-06-27T00:00:00Z&patient=Group/123

3 | Getting to Near Real-Time Without Breaking Rate Limits

Achieving near real-time healthcare analytics is not about polling faster—it’s about extracting the right data at the right cadence while respecting Epic’s API limits.

a) Tiered Extraction Frequency

Different healthcare datasets require different refresh intervals.

b) FHIR Subscriptions — The Key to Real-Time

Instead of polling continuously, register a FHIR Subscription so Epic pushes notifications whenever monitored resources change.

var subscription = new
{
    resourceType = "Subscription",
    status = "requested",
    reason = "Real-time vitals monitoring",
    criteria = "Observation?category=vital-signs",

    channel = new
    {
        type = "rest-hook",
        endpoint = "https://yourapp.azurewebsites.net/api/fhir-webhook",
        payload = "application/fhir+json"
    }
};

Once Epic sends a notification, your .NET webhook performs a single targeted FHIR read for the modified resource. This approach dramatically reduces API traffic compared to continuously polling thousands of patient records every few minutes.

c) Rate-Limit-Aware Client Design

Production-grade healthcare integrations should build resiliency directly into the .NET extraction layer rather than relying solely on infrastructure.

• Respect Retry-After headers and Epic’s published throttling limits.
• Implement Polly for exponential backoff with randomized jitter.
• Queue requests through Azure Service Bus to smooth request bursts instead of issuing large parallel batches.

var retryPolicy = Policy
    .Handle<HttpRequestException>()
    .OrResult<HttpResponseMessage>(
        r => r.StatusCode == HttpStatusCode.TooManyRequests)
    .WaitAndRetryAsync(
        5,
        retryAttempt =>
            TimeSpan.FromSeconds(Math.Pow(2, retryAttempt)) +
            TimeSpan.FromMilliseconds(new Random().Next(0,1000)));

d) Decouple Power BI from Epic Entirely

Recommended Power BI integration patterns include:

• Streaming Datasets via the Power BI REST API for operational dashboards such as Emergency Department census and bed occupancy.
• DirectQuery against Azure Synapse Serverless for near real-time analytical drill-through.
• Import Mode with scheduled refresh every 15–30 minutes for high-performance enterprise reporting.

// Push rows to a Power BI Streaming Dataset

var pbiClient = new HttpClient();

var payload = JsonSerializer.Serialize(new[]
{
    new
    {
        Timestamp = DateTime.UtcNow,
        BedOccupancy = currentOccupancy,
        Unit = unitName
    }
});

await pbiClient.PostAsync(
    streamingDatasetPushUrl,
    new StringContent(
        payload,
        Encoding.UTF8,
        "application/json"));

Summary

The foundation of a successful healthcare analytics platform is straightforward: extract data in bulk, stream only high-priority events, and isolate reporting from the operational EHR system. By separating Epic from the analytics layer, near real-time dashboards become a function of your Azure pipeline’s refresh cadence rather than a constant battle with API rate limits. This architecture delivers scalable, secure, and production-ready healthcare analytics while preserving the performance and stability of the Epic environment.