Databricks
Data Platform
Data lakehouse platform, DBRX model, Mosaic ML acquisition.
Recent activity
-
Databricks has developed an integration with AWS HealthLake, allowing users to query HealthLake-managed FHIR data directly in Databricks, bringing Databricks governance, analytics, and AI capabilities to clinical datasets. This integration uses OpenSharing to securely expose data in HealthLake without requiring any export pipelines or data duplication. AI summary
Read more → -
Databricks' Feature Views unify batch and streaming features for training and serving, enabling real-time personalization with 34ms p95 latency. A real-time next-best-offer recommender for retail banking combines stable customer context with recent clickstream activity, using the same feature definitions for training and serving. AI summary
Read more → -
Tabular foundation models (TFMs) utilize the transformer architecture to make predictions on new datasets without requiring bespoke training, accelerating predictive ML experiments. Databricks AI Runtime provides managed GPUs, MLflow tracking, and Unity Catalog integration to support experimentation and fine-tuning of TFMs. AI summary
Read more → -
Databricks has launched the AI Index, a publicly available resource that tracks enterprise AI model adoption, spending trends, and coding agent performance. The index combines anonymized usage data from 10,000+ companies using Unity Gateway with coding agent benchmarks based on real development tasks. AI summary
Read more → -
To create repeatable skills in Genie One, start by mapping the workflow for a specific task, considering factors like conditions, tools, data, and steps involved. Then, build the skill in Genie One through a conversation, describing the workflow in your own words and asking Genie One to interview you. Test the skill like Genie's first week on a new task, ensuring it delivers the desired output and correcting any issues in conversation. Govern the skill with Unity Gateway, a central platform for publishing, discovering, and governing skills, and share it with your team to make it reusable across the organization. AI summary
Read more → -
Databricks has been positioned as a Leader and highest in execution in the 2026 Gartner Magic Quadrant for Data Lakehouse Platforms, recognizing the company's pioneering work in the lakehouse market category. The lakehouse category, pioneered by Databricks, brings together data engineering, data warehousing, streaming, BI, and AI on a unified, open, governed foundation. AI summary
Read more → -
Apache Spark is generally considered the default choice for most teams due to its unified engine for batch, streaming, SQL, and AI, which can now achieve millisecond latency with Real-Time Mode. In a Databricks benchmark, Spark Real-Time Mode matched or beat Apache Flink on p99 latency for stateless, aggregation, and join workloads. Flink remains a better fit for specialized cases such as very large keyed state or complex event processing on an existing Flink platform. AI summary
Read more → -
OpenAI and Databricks have partnered to enable enterprises to deploy AI models that understand the business context and can act on it. This partnership combines OpenAI's frontier intelligence models (Codex, ChatGPT, and GPT models) with Databricks' governed data, business definitions, and access rules, allowing for three key enterprise adoption patterns: accelerating work for business users with ChatGPT and the Genie plugin, coding with enterprise-level governance through Codex and Unity Gateway, and transforming business processes with GPT models and Databricks Apps. AI summary
Read more → -
Databricks Real-Time Mode is a Spark Structured Streaming trigger that processes records continuously as they arrive, reducing latency to milliseconds. It's enabled by changing one trigger line and supports stateful operators, stateful aggregation, deduplication, and transformWithState. Real-Time Mode is generally available on Databricks and rolling out across open source Apache Spark releases, offering a faster alternative to Apache Flink for most workloads. AI summary
Read more → -
Lakebase's LTAP design stores data in a single, durable copy in the lake, using a hierarchy of caches and object storage to serve both operational data and analytics, eliminating the need for a costly CDC pipeline. AI summary
Read more → -
Constellation is building a governed data foundation on Databricks using Unity Catalog for lineage, access control, and shared business context, enabling unified governance and semantics across the enterprise. AI summary
Read more → -
Eclipse Aerospace simplified flight-data ingestion with Databricks by leveraging PySpark custom data sources, which enabled them to bring proprietary flight telemetry into Databricks without a separate ingestion platform. This approach reduced latency, cost, and operational overhead, while improving observability and engineering efficiency. AI summary
Read more → -
H6.ai uses Lakebase Postgres to build AI-powered financial automation by providing every customer with a dedicated database, isolating their financial data, and enabling frequent regression testing at scale through copy-on-write branches, disposable schema-migrated branches, and a panel of AI judges. AI summary
Read more → -
Lakehouse//RT scales real-time analytics without increasing costs by supporting high concurrency, faster query execution, autoscaling in small increments, and eliminating duplicate infrastructure. This approach enables customers to deliver real-time performance with sub-100ms latency and thousands of concurrent queries, while reducing costs by 6x to 10x compared to separate real-time serving layers. AI summary
Read more → -
Databricks Genie is now generally available in Slack, allowing users to get governed answers grounded in their business context without leaving the Slack workspace. Genie can update documents and tickets through MCP connectors after user approval, analyze files, and run analyses on a schedule. Governed by design, every question runs with the asker's permissions, and responses are visible to everyone in the channel by default, with the option to set responses to be private to the person who asked. AI summary
Read more → -
Databricks' Unity Gateway powers collaborative AI agents with Block's Buzz by providing centralized access to frontier and open-weight AI models, allowing employees to experiment with different combinations of models and agent harnesses to find what works best for each task. This flexibility enables teams to work with multiple agents in one place, without having to switch between different AI tools. By integrating Unity Gateway with Buzz, Block can efficiently manage and optimize AI costs and performance. AI summary
Read more → -
Here's a summary of the article for a developer/AI-ML audience: Detecting credential-driven attacks on Databricks involves identifying suspicious activity patterns that deviate from normal user behavior. A six-stage model outlines the typical progression of an attack: (1) gaining access with valid credentials, (2) mapping the environment, (3) harvesting more credentials, (4) finding a workable execution path, (5) establishing persistence, and (6) collecting data and cleaning up access. To detect these attacks, look for non-linear event sequencing, unusual activity patterns, and changes in user behavior. Utilize Databricks' verbose audit logs and the Detection App's Genie agent to spot these patterns. AI summary
Read more → -
A Lead Solutions Architect at Databricks spends their typical week balancing building tech, coaching colleagues, and partnering with executive management on business cases, often switching between coding, scaling applications, and collaborating with global teams on complex customer challenges. AI summary
Read more → -
Databricks has expanded its AI research and opened a new 235,000-square-foot headquarters in San Francisco, solidifying its commitment to the city as its home base for data and AI innovation. AI summary
Read more → -
Bright Machines captures PostgreSQL changes at the factory edge with Debezium, writes them as Apache Iceberg, sends them out over a single auditable outbound path, and reads them in place in Databricks through Unity Catalog federation. This setup enables customers to see live, governed dashboards of their factory data without opening any inbound access to the air-gapped plant. AI summary
Read more →