December 31, 2025Microsoft Fabric

Episode 314 – Microsoft Fabric November 2025 Feature Summary Part 2

Welcome back to Part 2 of the BIFocal Podcast’s coverage of Microsoft Fabric’s November 2025 feature announcements! Building on Episode 313‘s comprehensive look at platform updates and databases, Jason and John shift focus to the two workloads that power most Fabric initiatives: Data Engineering and Data Science. This episode delivers deep insights into spatial analytics, notebook improvements, user data functions enhancements, and the rapidly evolving data agents capabilities that are transforming how enterprises approach AI-driven analytics.

Data Engineering Innovations

Spark Connector for SQL Databases (Preview)

The conversation opens with practical news. Fabric developers now get native Spark support for SQL databases. Jason emphasizes the significance here. The Fabric runtime includes this connector preinstalled. Developers skip separate installation. Next, they run large-scale read and write operations. These operations happen directly from Spark notebooks. The target databases include Azure SQL Database. Azure SQL Managed Instance works too. SQL Server on Azure VMs connect seamlessly. Fabric SQL databases also join the party. Furthermore, the connector respects SQL engine-level security. Object-level security (OLS) applies. Row-level security (RLS) enforces permissions. Column-level security (CLS) protects sensitive columns. Jason uses this extensively with Azure SQL. He reports it “just works” with exceptional reliability.

ArcGIS GeoAnalytics for Microsoft Fabric (Generally Available)

John introduces what he calls “a bit niche” but incredibly powerful. ArcGIS GeoAnalytics for Spark reaches general availability. Microsoft and Esri partnered together. This partnership brings enterprise-grade spatial analytics directly into Fabric. You gain Spark notebooks with these capabilities. Spark job definitions also support spatial analytics. Work with geographic data? This feature changes everything. You can analyze location patterns now. You can track movement across time. You can perform spatial joins for enrichment. John walks through available analysis types. Hotspot identification and clustering are possible. Spatial pattern analysis across time works well. Location-based data enrichment happens seamlessly.

However, Jason emphasizes an important caveat. This integration isn’t free. You need a separate ArcGIS license. The Azure Marketplace charges $1,000 per month. This cost sits on top of your Fabric expenses. Therefore, budget accordingly for this feature. But here’s the upside. Organizations already using ArcGIS gain native Fabric embedding. This integration eliminates friction significantly. New possibilities for spatial data projects emerge immediately.

Progressive Notebook Rendering

Many Fabric users experience notebook frustration daily. Notebooks with large table outputs create problems. Loading takes forever. Now you can load them progressively instead. Previously, you waited. You couldn’t interact with your notebook. Every display() output needed completion first. Today, things are different. The interface renders outputs incrementally. You scroll down to see content. This improvement enhances the development experience significantly. Verbose output becomes manageable now. Large datasets no longer block your workflow.

Optimal Refresh for Materialized Lake Views (Preview)

John explains materialized lake views clearly. These views are system-managed queries essentially. They store results in a table. Fast access becomes possible this way. Think of them like Power BI semantic model refresh. You face a key decision. Should you refresh the entire table? Or should you use incremental refresh instead? The new optimal refresh toggle solves this dilemma automatically. It determines the most efficient approach. Jason notes the feature defaults to on. Furthermore, you can disable it if needed. Specific reasons might require this action.

Fabric Data Engineering VS Code Improvements

Fabric’s VS Code extension receives significant enhancements now. John admits the tool presents setup challenges initially. But these improvements address longstanding pain points. First, you open multiple notebooks. A single VS Code window accommodates them all. This reduces context switching significantly. Second, dependency management becomes simpler. You abandon Conda library reliance for basic functionality. Third, multi-tenant connection support improves. Organizations managing multiple Fabric environments benefit most. Finally, GitHub Copilot integration works seamlessly. Your Fabric notebooks gain AI-assisted code generation. You work with Fabric resources confidently. John finds this particularly exciting.

User Data Functions: The Ignite Edition

User Data Functions (UDFs) received substantial updates at Ignite. These updates continue their evolution remarkably. They matter as critical integration points. They integrate across the entire Fabric platform. These serverless Python functions encapsulate business logic. You invoke them from anywhere in Fabric. Call them from pipelines, notebooks, or Power BI reports.

Activator Integration

User Data Functions trigger directly from Fabric Activator rules now. This opens real-time event processing scenarios. When Activator detects conditions, it invokes a function. The function processes those events. Next, you pass event properties as parameters. Then you make intelligent decisions based on event data. Jason views this as powerful. Event-driven architectures become responsive. Your data reacts to events automatically.

Variable Library Integration

Jason identifies one critical capability. Accessing the Variable Library from UDFs matters greatly. He says it was “required in order to find it useful.” Variables now spread across Fabric. You find them in pipelines. You find them in dataflows. Now you find them in UDFs too. They become central to managing configuration. You store secrets without hardcoding them. This approach unlocks cleaner function development. Your code becomes more manageable.

Azure Key Vault Support

Security-conscious developers get direct Azure Key Vault access. You retrieve API keys securely now. Client secrets stay protected always. Other sensitive configuration doesn’t live in code. John notes this approach is standard practice. Azure development teams expect this. Bringing it to Fabric UDFs was essential. Production scenarios become possible finally.

Cosmos DB Support

User Data Functions connect to Cosmos DB databases now. You use Fabric-hosted versions or Azure instances. Native support makes this straightforward. You build applications that read from operational databases. You write back to them seamlessly.

Data Science: Beyond Machine Learning

Jason observes that “data science” has evolved significantly. It once referred specifically to machine learning. Model training was the focus. Now it’s become broader in scope. It encompasses AI functionality beyond Copilot. This evolution appears evident in feature updates. The focus shifts increasingly to AI integration. Data agents and agentic intelligence take priority.

Data Agents: Search Index and Foundry Integration

Data agents (Fabric’s conversational AI assistants) connect to Azure Search indexes now. They integrate with Microsoft Foundry as well. This dramatically expands reasoning capability. As a data agent creator, you add custom search indexes. You integrate with Foundry models seamlessly. Your agents surface insights from both sources. Structured tables feed data. Unstructured documents contribute knowledge. PDFs and other content enhance reasoning abilities.

Enhanced Data Agent Instructions

Working with data agents requires detailed instructions always. The November update increased instruction limits significantly. KQL database instructions expanded from 5,000 to 15,000 characters. That’s a 3x increase! Example queries jumped from 1,000 to 5,000 characters. That’s a 5x increase! Jason had encountered these limits before. He calls the 5x expansion “huge.” Finally, developers have enough room. You provide meaningful context now. You avoid being overly verbose.

AI Functions Enhancements

AI Functions received major expansions this cycle. You call these tools from notebooks. You call them from dataflows. They mark up your data intelligently. Beyond sentiment analysis, you get more. Beyond translation, there’s additional value. Now you summarize data. You extract key information. You classify content. You generate responses. These functions reduce payloads early. Developers work with smaller datasets downstream. Additionally, Microsoft expanded Foundry model integration. It now works in PySpark. You move beyond OpenAI-only models. Jason notes the Claude addition (Anthropic’s model). Gemini (Google’s) is available too. OpenAI remains supported. He compares the competition humorously. “This week this one’s ahead and this week that one’s ahead,” he jokes. The landscape shifts constantly.

MCP Server Support in Data Agents

Data agents leverage MCP (Model Context Protocol) servers now. This extends capabilities significantly. Your agents reach out to external MCP servers. They enhance reasoning substantially. They gather additional context. This context goes beyond enterprise data.

Data Agents Integration with Microsoft 365 Copilot

This is the feature Jason has been waiting for. Fabric data agents integrate directly with Microsoft 365 Copilot. You don’t need workarounds anymore! Previously, you built complex bridges. Foundry Studio hosted these bridges. Now the integration is built-in. Data stored in Fabric is directly accessible. Copilot accesses this data. You access Copilot in Teams. You access it in Office. Other M365 applications support it too. Jason hasn’t seen it available in his environments yet. However, he remains hopeful. This should roll out across tenants shortly.

Semantic Models and Data Agent Prep for AI

Power BI’s “Prep your data for AI” feature works with data agents now. Previously, you could prepare data for AI. Semantic models supported this preparation. But agents couldn’t use those preparations. Now they can! This unlocks planned scenarios. They were functionally incomplete before.

SQL Database and Warehouse Integration

Data agents connect directly to Fabric SQL databases now. Previously, they required intermediary lakehouses. Mirroring created these intermediaries. Now direct connections work. Simplified entry points exist for adding data sources. Lakehouse data sources integrate easily. Warehouse data sources join the platform. Creating data sources is now simple. Connecting data sources takes one click. The ribbon features this button prominently. This represents a significant usability improvement.

Machine Learning and Model Tracking Improvements

Data science teams get improved ML workflows. Foundational improvements apply to experiment tracking. ML artifacts benefit too. John notes this is specialized functionality. His primary focus lies elsewhere. But improvements matter for data science teams. You track machine learning models better. You manage model lifecycles more easily.

Internal Python Packages in ML Model Endpoints

AutoML endpoints now support internal Python packages. Flaml model endpoints support them too. You deploy machine learning models more easily. Custom dependencies integrate seamlessly. Model management becomes simpler.

Looking Ahead: Part 3 and Warehouse

Jason and John run out of time before covering Warehouse comprehensively. The Warehouse team provides thorough documentation always. They create excellent walkthrough content. These features deserve focused attention. This episode couldn’t cover them fully. Part 3 will provide detailed Warehouse coverage.

Key Takeaways

Episode 314 demonstrates Fabric’s maturation remarkably. The entire data engineering spectrum shows improvements. The complete data science spectrum advances too. The Spark connector brings native database integration. ArcGIS partnership opens spatial analytics. Most significantly, data agents expansion matters most. They demonstrate deeper reasoning. They support broader data sources. They integrate with M365 Copilot. Multi-agent orchestration becomes possible. Microsoft is investing heavily here. If you’re building with Fabric, you should prioritize data agents. Understanding them should top your list.

Resources and Further Reading

Part 3 coming soon: Jason and John continue their Ignite coverage. They’ll deliver a focused deep dive into Fabric Warehouse updates. Additional platform announcements complete the series.

Subscribe: SoundCloud | iTunes | Spotify | TuneIn | Amazon Music

Tags: AI, ArcGIS, Bifocal Podcast, data agents, Data Engineering, Data Science, Fabric, Feature Update, Ignite 2025, Microsoft Fabric, November 2025, Python, Spark, User Data Functions