Episode 304 – Fabric August 2025: Bursting Controls, Event Schema Registry & The T-SQL Magic Command

Recording early morning before hitting the road to Branson’s Collaboration Summit (with eight hours of car time ahead), Jason and John tackled the Microsoft Fabric August 2025 feature summary—a surprisingly hefty pre-FabCon release featuring Spark autoscale billing GA, bursting controls, event schema registry, Python notebooks for Fabric SQL, and what Jason declared his new favorite: the T-SQL Magic Command.

Conference Circuit Continues

With less than 48 hours until departure, the duo looked ahead to Branson (second-to-last conference of the year for Jason), followed by John’s final stop at CollabDays Ottawa, then their joint December finale at ESPC Ireland. Jason pitched the Teams workshop happening parallel to their Power BI/Fabric session—AC and Mark delivering Teams dev content for those with diverse team interests.

The Ozarks drive promised beauty, at least from Springfield onward. “Coming down from St. Louis, I don’t know that we get it quite as beautiful,” Jason admitted, though he hadn’t checked the route yet.

Deployment Pipelines Get Flat List View

The platform updates opened with UI improvements to deployment pipelines—adding a flat list view option alongside the existing workflow/lineage view.

“If you’re a lineage view person, sometimes it’s nice to be able to see this in a different way,” Jason noted, comparing it to workspace view preferences throughout Fabric.

John drew the parallel: “Whether you like living in the lineage view just in fabric or Power BI workspaces generally, or you like the list view, this would basically be the list view equivalent.”

Fabric API Specifications Centralized

Microsoft published OpenAPI specifications for all Fabric public APIs in a GitHub repository—providing “one-stop shopping for all of your Fabric API needs.”

Jason clarified for non-developers: “When you say swagger, you’re not talking about what the kids these days would think of it as—it’s a swagger file, right?”

John confirmed: “OpenAPI is the new name for what used to be called Swagger. It’s a file that has all the API definitions.”

Previously, developers chased definitions across various posts and sites. Now it’s centralized, consumable by tools that read these specifications, or browsable directly.

Cross-Tenant DevOps Support: The Big Deal

Service principal and cross-tenant support for Azure DevOps hit preview—”a bit of a big deal,” John emphasized.

Previously, workspace Git integration required DevOps and Fabric in the same tenant. GitHub offered more flexibility with simple authentication. Now DevOps matches that capability with:

• Cross-tenant authentication
• Service principal support (avoiding personal account dependencies)

Jason highlighted collaboration scenarios: “If you have consultancies that are working for you, or third-party vendors who are putting things into the environment… working with Power BI Tips and they had some code out there that you wanted to be able to integrate and keep on the main branch of stuff with them, that would be an opportunity.”

Spark Autoscale Billing Reaches GA

The data engineering section dominated with Spark updates, starting with autoscale billing reaching general availability. The feature addresses expensive Spark workloads consuming Fabric capacity units by executing on external resources.

“Spark can be expensive without realizing it because you’re running things in notebooks,” Jason explained, drawing the SQL analogy: “Like running select all from star without a top 1000—you didn’t want to see the entire database run.”

Autoscaling enables:

• Pay only for actual usage
• Automatic scale up/down based on demand
• Avoiding throttling from excessive CU consumption

Bursting Controls: Borrow Now, Pay Back Later

A new bursting control feature (on by default) lets intensive jobs use up to three times allocated CU for short periods, smoothing costs over time without throttling.

“Microsoft’s basically lending you a few, you pay ’em back with your excess capacity throughout the day,” John explained.

Critical distinction: bursting doesn’t apply to autoscale billing—they’re separate mechanisms. Autoscale runs on-demand outside the capacity; bursting temporarily borrows from available CU.

When to disable bursting: high-concurrency scenarios with multi-user notebooks. “You don’t wanna rob Peter to pay Paul in that situation,” Jason noted. One user’s burst could starve others’ capacity.

The pair acknowledged confusion between the two features, spending time clarifying their relationship: autoscale = pay separately for external resources; bursting = temporarily borrow from your own allocated CU pool.

Notebook Enhancements Across the Board

Multiple notebook improvements arrived:

• Job Insight diagnostic library: Java-based analysis tool for Spark job performance (preview)
• Enhanced monitoring for Spark high concurrency: Core UI improvements independent of the diagnostic library
• Pandas support for user-defined functions: Previously Spark-only, now accessible from lighter Python notebooks via Apache Arrow
• Notebook snapshots: Version control showing code state at execution time, regardless of invocation method
• Test capabilities for UDFs: Try functions before publishing
• OpenAPI spec generation: Create invocation specs for external calling or AI description

Jason discovered validation functionality firsthand while testing fast copy from Azure SQL: “You do have to save in order to do it first… once you’ve done your validation and get a successful validation, you can choose to run it.”

The T-SQL Magic Command

John introduced what became Jason’s new favorite: Python notebooks can now read/write Fabric SQL databases using “T-SQL Magic Command.”

“I do enjoy the fact that it’s called T-SQL Magic Command,” Jason declared. “Maybe my new favorite command name.”

The feature embeds SQL within Python notebooks via the Magic SQL library, enabling notebook experiences against Fabric SQL databases.

Jason immediately tested the companion feature—opening databases in SQL Server Management Studio directly from Fabric. Previously painful with SSMS 21’s “19 or 37 or maybe 2,942 different auth options,” the new button launches SSMS pre-authenticated.

“It passed me right through, much better experience,” Jason reported. “This was definitely a problem before and now it is not.”

Event Schema Registry: Big Deal for Real-Time

Real-time intelligence gained event schema registry (preview)—addressing Event Streams’ historical pain point of schema inference from first data batches.

“If your first set of data isn’t representative, you may wind up not being able to handle your data properly,” John explained.

The registry enables:

• Imposing schemas on event streams
• Saving derived schemas for reuse
• Managing schemas from Real-Time Hub
• Preventing issues from inaccurate inference

John called it “a big deal” without wanting to “make light of it,” noting the fundamental improvement for production scenarios.

Additional real-time updates included:

• Auto-discover schemas from Azure SQL CDC sources
• Database tree in realtime dashboards: Explorer showing all connected data sources during tile editing
• Simplified Azure Monitor connections: No more hunting for resource IDs
• Query sharing in KQL query sets: UI for distributing queries to colleagues
• Accelerated shortcut controls: Granular management of what data gets accelerated and how far back

Data Warehouse Logging Gets Visual

Warehouse audit logging, available since April, gained a visual experience—no more manual log review. Users can enable/disable logging and query through logs directly from Fabric UI.

Microsoft also renamed operations for clarity: “Create Data Mart” became “Create Artifact” with different friendly names for different artifact types, enabling faster grouping and filtering.

“Show me all the updated artifacts—Update artifact actually contains about 10 different types,” Jason explained. “You’ll be able to tell different when you’re doing the grouping by column.”

Data Agent Capacity Boost

Data agents gained support for larger data sources—previously capped at smaller sizes, now handling 100+ columns/measures and 1000+ tables.

“Some of my stuff has upwards between 250 and 500 tables to them,” Jason noted. “This will play nicely in that space as long as my number of columns are contained nicely.”

John acknowledged: “Your favorite things these days, data agents have got a boost.”

Copy Job Improvements

Copy jobs (distinct from pipeline copy activities) gained critical features:

• Reset incremental jobs: Force full copy instead of just changes
• Auto table creation on destination: Previously pipeline-only, now available for Lakehouse tables, SQL Server, Azure SQL, Fabric SQL, Snowflake, and Azure SQL Managed Instance
• JSON format support: Another pipeline feature graduating to copy jobs

“Things that Fabric talked to,” Jason summarized the broad destination support.

Pipeline Rebranding & Trigger Management

Microsoft renamed “data pipelines” to simply “pipelines”—prompting John’s deadpan: “I’m not sure how that’s less confusing given that we have deployment pipelines.”

“Microsoft is the best at naming things, John,” Jason replied.

Trigger management gained a dedicated ribbon button, and the template gallery added category filters for data sources and destinations.

Copilot for Get Data

Data source setup with copilot’s new “get data” capability (in preview for months) enables natural language data retrieval—tell copilot what you want, it fetches it.

Jason recalled Alex Powers demonstrating the feature, while John highlighted transformation descriptions: “Go build a date table for me and it will, given the parameters you want to tell it.”

No Dedicated AI Section This Month

As they wrapped, Jason noted the structural shift: “I didn’t even see an AI section this month… used to be that we had a section at the top that was about copilot and AI.”

The discussion touched on Fabric’s expanding scope—Power BI having its own blog, data engineering becoming their primary playground, gravitating toward different areas as the platform grows.

“Fabric is so big,” Jason reflected. “Data engineering in here is definitely the area that I tend to play in a lot more these days.”

Looking ahead to FabCon, they speculated about holdbacks: “Maybe it’s a hold back, right? Because next month being a FabCon month.”

With limited time before hitting the road (and John taking October off for adventures), they acknowledged going long but covering necessary ground. Jason teased guest hosts during John’s absence, reuniting around Build timeframe.

The August drop proved surprisingly robust for a pre-conference month—fit and finish features with genuine utility, from bursting controls to schema registries, wrapped up with the charm of a T-SQL Magic Command.

---

Links:

• Microsoft Fabric August 2025 Feature Summary
• Fabric API Specifications on GitHub
• Fabric Roadmap
• Episode 303 – Mike Carlo on Power Designer, Semantic Models & The Future of Embedding

Subscribe: SoundCloud | iTunes | Spotify | TuneIn | Amazon Music