Release Notes 2024.3 (General Availability)

Note

The release notes on this page reflect changes to the Alation application. If you’re looking for OCF connector release notes, jump to OCF Connector Release Notes.

RELEASE 2024.3.3

build 19.3.0.92673

Released December 4, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

• Lineage

• Domains

• Authentication

• Compose

• Search

• OCF Connectors

• Enhancements

• Bug Fixes

• Known Issue

Lineage

View Lineage Incrementally (Beta)

Alation Cloud Service on the cloud-native architecture

Previously, very large Compound Layout charts weren’t loaded due to their size. We’ve now introduced the ability for users to view these charts incrementally by adding nodes in batches. Incremental lineage view is available in both New and Classic User Experiences.

• Configuration by Admins:

  • Server or Catalog Admins can configure batches for increments in Admin Settings > Lineage Settings page > Load Settings. The Initial Vertical Nodes (Depth) setting, previously only applicable for the Classic lineage view, has been extended to the Compound Layout. This setting now determines the number of nodes displayed incrementally before the Show More button appears.

• User Experience:

  • When attempting to view an oversized Compound Layout lineage chart, users will be prompted with a warning dialog. This dialog explains that the chart is too large and provides options for proceeding. To view the lineage incrementally, users select Continue with Incremental Lineage. This mode allows for expanding the chart node by node using the Add (+) icons and providing easier navigation of complex data relationships. (AL-177716)

Show Schema Name on Parent Nodes

For Compound Layout lineage charts, we’ve added the ability to show schema names on the nodes which have a schema (RDBMS objects). Server Admins can enable this feature using the alation_conf flag alation.feature_flags.enable_parent_node_schema_name. (AL-177474)

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

Cross-Source Lineage Generation Uses the JDBC URI

We’ve expanded the functionality for handling host and port data in cross-source lineage configurations. Now, the system can automatically retrieve this information from the connector settings. This update makes the Additional datasource connections field optional for connectors that automatically provide the necessary URI. The Additional datasource connections field is required only when the connector does not populate the JDBC URI, such as with API-based connectors like Salesforce or Google BigQuery. (AL-172918)

Domains

Search for Objects Without a Domain

We added another selection in Search within the Domains dropdown, Not Set. Selecting this option enables users to filter and view catalog objects that are not associated with any domain. This feature is intended to help identify and manage uncategorized objects within the catalog. (AL-121322)

Enhanced Domain Propagation Transparency

We’ve improved the visibility of domain assignments with a new update in the Audit Domain Assignment dialog available from the Domains built-in field on catalog pages. Users can now clearly see when domains are not propagated to objects due to conflicting rules: The dialog now includes an additional table, Domains not assigned. The Domains not assigned table displays domains that, while assigned to parent objects, have not been assigned to the object you’re viewing. Each entry in this table includes a Reason for non-assignment, which explains why specific domains weren’t propagated. (AL-172652)

Authentication

User Interface for Authentication Service Management in the Agent

Alation Cloud Service instances on the cloud-native architecture

Installing, updating, and uninstalling Authentication Service on Agents for the retrieval of secrets from vaults is now supported through the user interface (UI). Admins can also check the status and version of their Authentication Service that was installed via the UI on the Agent detail page for improved oversight. (AL-154367)

Compose

Hierarchical Navigation Panel (Beta)

A new navigation panel view has been introduced in Compose that presents a hierarchical listing of schema, table, and column objects and allows for resizing of the entire navigation panel. The feature is currently under the alation_conf flag alation.compose.enable_tree_sidebar.

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

• Default: False. When enabled, the current navigation panel is replaced with the new version. (AL-101634)

• To test the feature, you can display or hide the new navigation sidebar directly in the user interface. To display it, add the following parameter to a Compose URL: ?enableTreeSidebar=true, for example <Base URL>/compose/query/{queryId}/?enableTreeSidebar=true. To disable the new navigation sidebar, in a similar way, pass ?enableTreeSidebar=false in the URL: <Base URL>/compose/query/{queryId}/?enableTreeSidebar=false.

Search

Default Top Ranking for Exact Match Queries Without Quotes

Exact matches without double quotes now automatically rank at the top by default, enhancing Search accuracy. Previously, a search string without double quotes that consisted of multiple words was broken down into separate tokens for searching, and search results might rank partial matches of the search string at the top due to various factors. With this change, objects with content exactly matching the search string will rank higher than objects with partial matches. If Server Admins find that this default behavior adversely affects other search types, they have the option to adjust the exact match boost factor to better balance Search result rankings. (AL-174607)

OCF Connectors

Certified Support for PostgreSQL Versions 16 and 17

The OCF connector for PostgreSQL supports database versions 16 and 17. (AL-153605)

Value Title Lookup Now Works with Dynamic Profiling

The Value Title Lookup feature has been updated to support both enabled and disabled states of Dynamic Profiling:

• Dynamic Profiling On: When dynamic profiling is enabled, the Value Title Lookup preview will use dynamic profiling user credentials.

• Dynamic Profiling Off: When dynamic profiling is disabled, the Value Title Lookup preview will use the credentials specified in the data source settings under General Settings. (AL-131988)

Enhancements

• Document Hub subfolders and navigation links are now displayed in the left-side navigation panel. (AL-166618)

• Server Admins now have control over the Semantic Search part of Intelligent Search using the allow_semantic_search flag in alation_conf, which is set to True by default. For an Alation Cloud Service instance to enable Semantic Search, both the entitlement flag and the allow_semantic_search flag must be set to True. Semantic Search will be disabled if either flag is set to False. The allow_semantic_search flag specifically allows Server Admins to turn off Semantic Search if it isn’t permitted on the instance, regardless of the instance’s entitlement to Intelligent Search capabilities. (Alation Cloud Service on the cloud-native architecture, AL-174771)

• The Alation Help Search dialog has replaced the Visit Help Center link in the Help menu (identified by the question mark icon (?) on top right) within the New User Experience. This update is intended to streamline access to support resources directly through a searchable interface, enhancing user assistance. (Alation Cloud Service on the cloud-native architecture, AL-176648)

• For the Classic User Experience, we improved the data dictionary upload functionality by locking specific objects instead of the entire data source. Previously, uploading a dictionary for any object would lock the entire data source, preventing simultaneous uploads for different objects by different users. The new locking mechanism targets only the object being updated, rather than the entire data source. This change allows concurrent dictionary uploads for different objects within the same data source. (AL-177370)

Bug Fixes

• Previously, searching for a term in the terms table did not prioritize it at the top of the search results, and using exact search terms enclosed in quotation marks retrieved no results. We have refined the search functionality within the Document, Folder, and Navigation Link tables. Now, table searches will focus solely on the Title field. The updated search logic prioritizes prefix matches, displaying them at the top of the results list, followed by substring matches. (AL-168729)

• Fixed an issue where advanced search filters exhibited inconsistent behavior after enabling the Document Hub feature flags. (AL-167395)

• Corrected an issue where terms created with private access settings were incorrectly appearing in search filters. Now, private terms adhere to their access restrictions and do not show up in public or unauthorized searches. (AL-147039)

• Fixed an issue where refreshing the upload data dictionary page from any page other than the data source resulted in a 404 error. We’ve updated the Django URL pattern that previously restricted dictionary uploads to only the data source page. This change now permits uploads from data source children pages, such as schemas and tables. (AL-178438)

• Fixed a bug where tags were not appearing on the columns page of a data source, despite being added to the column’s custom template. We have modified the conditions used to filter Snowflake tags on catalog pages, ensuring that tags now display correctly across all relevant pages. (AL-174656)

• Fixed an issue where tag and tag link ingestion extended to 5-6 hours for 1 million links after upgrading to Snowflake connector version 2.3.4. We’ve optimized the ingestion process, reducing it to under 2 hours. Enhancements include: reusing tag details across all link ingestions to avoid repetitive data retrievals, preventing redundant tag additions to templates, updating modified tag links only, minimizing data processing. (AL-161720)

• Fixed an issue where after updating the Hive connector to version 1.3, executing a query that returns zero rows in Compose inaccurately displayed the message -1 row(s) affected. This message could confuse users, as such queries previously showed an empty result set. After this fix, the system will clearly indicate Retrieved all 0 rows. Additionally, for Hive OCF connectors using Query Service, we’ve corrected an issue where attempting to fetch Hive logs triggered errors due to unsupported features. The system now bypasses log fetching when Query Service is enabled, ensuring compatibility. (AL-163630)

• Fixed an issue where the catalog sets page experienced an AWS LB Gateway timeout when attempting to load over 5,000 catalog sets. Resolved by optimizing the loading process for catalog sets, ensuring accessibility even with a large number of objects. (AL-169847)

• Extract jobs were failing in Alation Analytics due to an error Generic exception occurred in custom_field_helper that was triggered by missing timestamps in the source tables for custom field values. Fixed by adding a validation check to ignore timestamps in custom field values if they are not present in the source tables. (AL-177890)

• Fixed an issue where users encountered 404 errors when uploading dictionaries through the version 1 data dictionary user interface (UI) for all objects except the data source. This problem was caused by an optimization that only locked the source object, not supporting the deprecated version 1 flow. Now, if a user tries to upload a dictionary using the version 1 UI with the data dictionary redesign flag disabled, they will be automatically redirected to the data source dictionary upload page, ensuring compatibility and preventing errors. (AL-178980)

• Fixed an issue where automated extractions from a file system source were experiencing a Job not running error after upgrading to 2024.3.1, while manual extractions completed successfully. The issue was identified in the auto-synchronization process of the Celery Beat component. This update implements logic ensuring consistency in creating Celery tasks between automated and manual extractions. (AL-178264)

• Fixed an issue where users experienced an error Sorry, there was an error in Alation when attempting to navigate to any article. This issue was linked to the disabling of the alation.feature_flags.enable_permissions_middleware flag. Addressed the issue by preventing the error dialog from appearing when this feature flag is disabled. (AL-178244)

• In the Classic User Experience, the source selection dropdown in the lineage chart preview was non-functional. The dropdown failed to trigger an API call necessary to fetch the list of sources. This fix ensures that selecting a source properly initiates the API call to fetch and display the relevant source data on the lineage chart. (AL-177883)

• Fixed an issue where users could access text logs for queries even if they did not have access to the parent data source. Resolved by ensuring that users cannot view query logs unless they have the appropriate permissions for the associated data source. (AL-177101)

• Fixed a bug where in the Classic User Experience, when navigating from Browse on a domain’s page to the full-page Search page with the Federated Catalog feature enabled, the Search incorrectly displayed only that specific domain. This occurred regardless of previous selections of My Domains Only or Full Catalog. The fix ensures that the correct domains are selected on the search page, reflecting user choices. (AL-176555)

• Fixed a security vulnerability reported by a customer. (AL-175789)

• Fixed an issue where users experienced error messages like page takes too long to load when attempting to retrieve Impact Analysis with larger max distances, even before the expected timeout period. This error was related to the enabling of granular object privacy settings. This fix ensures that the Impact Analysis report performs efficiently, without premature timeouts. (AL-174890)

• Fixed an issue where users received health check emails indicating that no backup jobs had finished as expected. With the transition to using RDS snapshots for backups, we have updated our systems and removed the alerts for backup jobs previously managed by Celery. The feature flag that enables email alerts for Celery jobs will no longer trigger alerts for backup job failures, aligning with our current backup infrastructure. (Alation Cloud Service on the cloud-native architecture, AL-174745)

• Fixed vulnerabilities allowing the upload of SVG files potentially containing malicious code. We have updated our security protocols to include a more comprehensive list of script functions that are now forbidden in SVG files uploaded to JSON or UI-based homepages and co-branding areas. (AL-173035)

• Fixed an issue where the author and query title were missing from the query results sharing dialog. The issue arose during the transition from the Mustache template engine to Handlebars, as the template file was not updated to align with the new syntax. Fixed by migrating the template file to the Handlebars syntax (AL-171779)

• Fixed an issue where Search results were not displaying expected results for BI objects. (AL-164288)

• Fixed an issue where the BI Folder API (/bi/server/{server_id}/folder/) was not returning reports, despite indicating the correct report count. After the fix, the endpoint now accurately displays all reports associated with the folder. (AL-164125)

• Fixed a bug where changes to the title of a schema were not consistently reflected in the Title history. For example, the history didn’t record instances where titles were completely removed or cleared (changed from non-blank to blank). After the fix, all modifications to Titles, including changes from non-blank to blank, are accurately recorded in the Title history. (AL-159728)

• Fixed a bug where term names containing the ampersand (&) character were incorrectly displayed as & in search filters and data dictionary downloads. After the fix, the system will unescape special characters in term names when they are used in custom fields on catalog pages and in the Search filters. This update ensures that term names are displayed correctly across the platform. (AL-156290)

Known Issue

Folders that contain exclusively navigation links do not expand in the left-side navigation panel, making any linked documents within these folders not visible from the left menu. As a workaround, ensure that each folder contains at least one child document along with any navigation links. This temporary solution allows the folder to expand normally and makes all linked documents visible. (AL-179106)

RELEASE 2024.3.2.3

build 19.2.3.92412

Released November 7, 2024

Note

This patch release is available for Alation Cloud Service instances on the cloud-native architecture. This patch release doesn’t apply to customer-managed (on-premise) instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

• Fixed an issue where automated extractions from a file system source were experiencing a Job not running error after upgrading to 2024.3.1, while manual extractions completed successfully. The issue was identified in the auto-synchronization process of the Celery Beat component. This update implements logic ensuring consistency in creating Celery tasks between automated and manual extractions. (AL-178264)

RELEASE 2024.3.2.2

build 19.2.2.91714

Released November 7, 2024

Note

This patch release is available for Alation Cloud Service instances on the cloud-native architecture. This patch release doesn’t apply to customer-managed (on-premise) instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

• Fixed an issue where lineage migrations didn’t run after the update to 2024.3.2, which made lineage charts unavailable in the user interface. (AL-178054)

Known Issue

• When a new tenant is created (except for free trial environments), the default Business Glossary displays with an empty Title and Description as an Untitled Glossary. Users can update these fields, and the information will save correctly. The issue only affects completely new deployments. (AL-176554)

RELEASE 2024.3.2

build 19.2.0.91171

Released November 5, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

In this release:

• Upload Logical Metadata (ULM) API Deprecation Schedule

• Document Hubs

• Workflows

• New User Experience Updates

• Compose

• Search

• ALLIE AI Suggested Descriptions

• Lineage

• Alation Analytics

• OCF Connectors

• Alation Chrome Extension

• Bug Fixes

Upload Logical Metadata (ULM) API Deprecation Schedule

• Updated Timeline for the ULM API Deprecation

• Risks with Using the ULM API on Alation Cloud Service

Updated Timeline for the ULM API Deprecation

Note

This announcement is also available on the Alation Community: Changes to Upload Logical Metadata (ULM) API Deprecation and Sunset Timeline (requires a login).

The deprecation of the Upload Logical Metadata (ULM) API was initially scheduled with the 2024.3.1 Alation release (October 2024). In response to customer feedback, the deprecation of this API has been postponed.

• New Dates:

  • Deprecation: release 2025.1 (March 2025)

    • Deprecation—The API remains accessible to users but will no longer receive new features or bug fixes.

  • Sunset: release 2025.3 (September 2025)

    • Sunset—The API is no longer supported, is removed from the product, and no longer functions.

• Why Have the Dates Been Changed?

  Due to customer feedback regarding the lack of alternatives for updating article custom field values, we’ve adjusted the deprecation schedule for the ULM API to ensure a smoother transition. The deprecation will now take effect only after the following tools are generally available (GA):

  • Document Hubs GA: Release 2024.3.2 (November 2024).

  • Article to Document Hub Migration Solution GA: Scheduled to be released with 2024.3.3 (December 2024).

• What to Expect:

  • Alation doesn’t recommend using the ULM API.

  • Following the 2025.1 release, Alation will no longer prioritize support for the ULM API and may decline support requests related to this API.

Risks with Using the ULM API on Alation Cloud Service

Note

This announcement is also available on the Alation Community: Risks with the Upload Logical Metadata (ULM) API on Alation Cloud Services (ACS) (requires a login).

Alation Cloud Service customers using the ULM API may encounter issues where updates to Titles, Descriptions, or custom field values do not appear correctly in the Alation user interface.

• Understanding the Risk:

  Alation is enabling Logical Metadata Services for all Alation Cloud Service customers. This new infrastructure aims to enhance performance and scalability for metadata elements such as Titles, Descriptions, and custom field values. With this change, the Alation UI serves content from Logical Metadata Services instead of the legacy logical model. However, updates made via the ULM API are applied only to the legacy model and are not compatible with Logical Metadata Services.

• I am using the ULM API on Alation Cloud Service. What are my options?

  • Alation doesn’t recommend using the ULM API. This API is planned to be deprecated with the 2025.1 release (March 2025). Use alternative APIs instead.

  • Contact Alation Support and request a temporary opt-out of Logical Metadata Services enablement. The temporary opt-out can be granted until the API deprecation with the 2025.1 release (March 2025).

• I am using the ULM API on a customer-managed instance. What are my options?

  • Alation doesn’t recommend using the ULM API. This API is planned to be deprecated with the 2025.1 release (March 2025). Use alternative APIs instead.

  • No action is required regarding the Logical Metadata Services enablement as they aren’t available on customer-managed instances.

Document Hubs

Document Hubs Are in General Availability (GA)

Document Hubs are now in GA on both Alation Cloud Service and customer-managed deployments. This modern documentation feature set enables the creation of customized documentation objects. It supports a shared metadata language and semantics tailored to your organization’s specific needs. Document Hubs include the Glossary of Terms, an out-of-the-box documentation object available in the catalog.

Important

There is no immediate impact on the use of articles. However, users should be aware of the upcoming milestones related to this transition:

• Release the Article Migration Solution:

  Migrating existing articles to Document Hubs is considered mission-critical. We have developed an in-product migration solution and additional resources to enable customers to transition at their own pace. This migration solution will be introduced through a tightly managed pilot with selected customers in November and is scheduled to become generally available by December 15th, 2024.

  With the Migration Solution in GA, we recommend that customers plan to use this tool to consolidate all content under Document Hubs.

• Enable Document Hubs by Default

  In release 2025.1 (March 2025), we plan to enable Document Hub-related feature flags by default, establishing Document Hubs as the primary solution for data documentation.

• Sunset Articles

  Support for articles and the related functionality is scheduled to be discontinued starting with release 2025.3 (September 2025). Further details will be provided as the date approaches.

Document Hubs Hierarchy

Document Hubs now support a robust content hierarchy, enabling the creation of nested folders and establishing parent-child relationships among documents using sub-folders and sub-documents. Both folders and documents can be moved within a Hub, allowing for flexible organization. Each document now has a single primary parent folder, simplifying management through streamlined access settings and workflows. Users can create navigation links to multiple other folders for enhanced accessibility. Document templates now feature a component that enables the display of a document’s child items. These child documents can be presented as a list in the right-side panel or displayed in a table within the main panel for improved content organization.

Document and Term Public APIs Support Document Hub Hierarchy-Related Properties

Added new parent_document_id and nav_links_folder_ids properties to the Document and Term public APIs. (AL-175354)

Workflows

Support for Document Hubs in the Add Object Request and Change Request Workflows

• Added the Add Object Request workflow support for adding new documents to a Document Hub. (AL-168892)

• Added workflow support for changing Document Hub objects (documents and folders). ​​(AL-168891)

Ability to Assign People Sets as Reviewers

Added support for selecting People Sets as Reviewers in addition to Users and Groups to the Add Object Request workflows. While configuring Add Object Request workflow for document objects within a Document Hub, an admin user can select a People Set field as Reviewer. When a new document is added to the folder enrolled in the workflow, this People Set will be the reviewers for the new document. Similar to user and group reviewers, People Set reviewers will receive email notifications for the Add Object Request workflows and the review requests will be listed as Tasks in their Inbox. (AL-169046)

Workflows Public API Supports Document Hub-Related Properties

The response from the GET method of the Workflows public API now includes the information about the Document Hub for document and folder objects if the workflow applies to these types. (AL-169049)

New User Experience Updates

Alation Cloud Service instances on the cloud-native architecture

• Migrated OCF RDBMS data source pages to New User Experience: Data Source Admins using New User Experience will now see the RDBMS data source settings in New User Experience for any data sources where the connector supports. (AL-171767)

• Added the API Resource object page to New User Experience. (AL-169640)

• Added support for object previews on mouse-over over an object link. (AL-166547)

Compose

Compose Is Compatible with All Alation-built RDBMS OCF Connectors on Alation Agents

Alation Cloud Service on the cloud-native architecture

Query Service is now automatically enabled for all connectors installed on an Alation Agent. This eliminates the need for admin users to submit Support tickets to activate Query Service for connectors when setting up data sources. (AL-174576)

Compose Supports Kerberos Authentication with the Hive OCF Connector

Users of Compose on both Alation Cloud Service and customer-managed instances can soon start using Kerberos authentication through the Hive OCF connector, including support of keytabs. This change ensures support in the Alation platform and facilitates the migration of Hive sources from native connectors to OCF or the migration to the Alation Cloud Service. Necessary updates to the Hive connector will be included in an upcoming release of the connector. (AL-164036)

Search

Previously, Alation Search would tokenize search strings containing underscores, such as apple_pie, into separate tokens (apple and pie). Tokenization could yield search results that were unrelated to the full search string. Now, Search treats strings containing underscores as single tokens. This change aims at reducing unrelated search results and enhancing the accuracy and relevance of searches. (AL-173913)

ALLIE AI Suggested Descriptions

Alation Cloud Service on the cloud-native architecture

License Tab Displays Usage of Suggested Descriptions

The License page in Admin Settings now displays the number of ALLIE AI Suggested Descriptions included with the license. (AL-171581)

Note

Usage tracking will be introduced in a future release.

Lineage

Lineage Graph Schema Filtering (Beta)

Users can now filter RDBMS objects like tables and columns by their schema. The feature improves large graph navigation and experience by providing the ability to remove redundant metadata from lineage charts. (AL-166131)

Bulk Impact Analysis and Upstream Audit (Beta)

Users can now download the Impact Analysis and Upstream Audit reports in the CSV format from the catalog pages of the RDBMS data source, RDBMS schema, and BI server objects. Click on the three-dot menu in the top-right corner of the catalog page. From the dropdown menu, select either Impact Analysis or Upstream Audit to open the export dialog. Then use the dialog to configure the export. The reports are generated asynchronously. When ready, you’ll receive an email with a link to download the file. (AL-165289)

“Graceful Exit” from Large Lineage Charts

On the Lineage tab, users will now see a “graceful exit” dialog when they interact with graphs that exceed the configured chart size threshold. The dialog allows users to navigate to the Impact Analysis and Upstream Audit lineage reports instead of displaying the chart. Users can also select a specific column of a parent node to load lineage for that column only. (AL-172516)

Lineage for Stored Procedures (Beta) Enabled by Default

Alation Cloud Service instances on the cloud-native architecture

Lineage for SQL-based stored procedures is now enabled by default, with the alation_conf parameter alation.ocf.mde.rdbms.parse_function_ddl.enabled set to True. This capability has been expanded beyond SQL Server to include additional data sources: Snowflake, Amazon Redshift, IBM Db2, Greenplum, SAP HANA, MySQL, Oracle, PostgreSQL, Sybase, Teradata. (AL-175889)

Note

There are still a few known issues in this beta feature, with fixes planned for future releases:

• The lineage chart may not be accurate when queries include columns with unqualified names. (AL-170059, AL-170264)

• Object resolution fails for queries with long object names. (AL-176370)

• Lineage isn’t generated for nested stored procedures PostgreSQL. (AL-176370)

GSP Upgrade

The General SQL Parser (GSP) component has been upgraded to version 3.0.x.x. This upgrade takes advantage of new features and bug fixes in GSP and adds support for parsing queries against Snowflake dynamic tables to the Alation platform. (AL-167410)

Note

Necessary updates to the Snowflake OCF connector are planned for an upcoming connector release.

Alation Analytics

Support for Document Hubs in Alation Analytics

Added columns logical_group_id and logical_group_type to the workflows table in Alation Analytics. These new columns allow filtering Document Hub-related workflows. For workflows not related to Document Hub objects, these columns will have NULL values.

• logical_group_id—ID of a logical group. For Document Hubs, it’s the Document Hub ID.

• logical_group_type—Type of a logical group. For Document Hubs it’s document_hub. Currently, document_hub is the only applicable logical group type. (There may be additional logical group types such as domain in the future.) (AL-169048)

Enhanced Handling of Deleted Information

To ensure data consistency between Alation Analytics and the catalog, we’ve enhanced the ETL process to capture changes not only when records are updated (ts_updated) but also when they are deleted (ts_deleted) for more object types. This enhancement applies to extraction jobs related to content (articles, catalog set membership, custom glossaries, and conversations), workflows, and policies. The extraction queries for these jobs will now include filters for the ts_deleted and deleted columns, ensuring that deletions are accurately reflected in Alation Analytics. (AL-164485)

Performance & Stability Improvements to Alation Consumption Tracker (ACT)

Alation Cloud Service instances on the cloud-native architecture

The improved ACT pipeline now supports 4 times more data extraction per run. The updated limit is ~120 million records for one month of historical data, as opposed to the previous limit of 30 million records for the same duration. Additionally, we introduced a range of enhancements to the extraction and processing of the qli_events table, including configurable scheduling, ID-based extractions, and dynamic record limits. Replication to Snowflake is now supported for this table to ensure access in third-party BI tools. ACT must still be enabled via a backend configuration, and the Alation Analytics merge pipeline must be operational for full functionality.

A new lighter schema has been introduced that maintains existing functionality with enhanced performance & stability. Full updated schema is described below. (AL-159051)

Column Name	Type	Description
dim_load_id	integer	Not nullable
dim_ts_created	Timestamp with time zone	Not nullable
dim_checksum	bigint	Not nullable
id	bigint	Not nullable (Generated by identity)
execution_id	bigint	Not nullable
session_start	Timestamp with time zone	Optional
ds_id	integer	Optional
db_account	text	Optional
seconds_taken	double precision	Optional
total_cpu_time	double precision	Optional
num_result_rows	bigint	Optional
canceled	Boolean	Optional
session_id	integer	Optional
tables_queried	text	Optional
columns_queried	text	Optional
query_fp	bigint	Optional
executed_at	timestamp with time zone	Optional

OCF Connectors

Updates to the Open Connector Framework (OCF) connectors are released every week. See OCF Connector Release Notes for information on the updates in the month of October 2024. A few October highlights are listed below.

OCF Connector for SSRS Now Supports Table and Column-Level Lineage

Table-level lineage is supported from connector version 1.3.0 and is certified with SQL Server only. Table-level lineage is not supported for stored procedure-based datasets. Column-level lineage is available only if the dataset and report column names match. (AL-171523)

Looker OCF Connector Enhancements

Starting with version 2.1.0, the connector catalogs LookML Models as BI Folders, Explores as BI Datasources, and Looks fields. Each Explore object is cataloged as a BI datasource and has its own Fields and Connections. (AL-164600)

Hive OCF Connector

Alation Cloud Service instances on the cloud-native architecture

The Hive OCF Connector installed on an Alation Agent now supports Compose. (AL-175152)

Cloud-Native Architecture for File System Sources

Alation Cloud Service instances on the cloud-native architecture

• The ingestion services used by metadata extraction and column extraction jobs of file system sources will be moved to the cloud-native architecture.

  Note

  This feature is disabled by default. It’ll be rolled out in phases to all customers across different regions.

• Added support to remove columns as part of column extraction if a column is removed from a file in the file system. This feature is available only when the cloud-native architecture is enabled for file system sources.

Alation Chrome Extension

Alation Cloud Service instances on the cloud-native architecture

Two versions of the Alation Chrome Extension were released in October 2024:

Version 1.5.1

Released October 28, 2024

• Added Japanese localization to enhance accessibility for Japanese-speaking users, providing a localized browsing experience. (AL-173138)

• The Alation Chrome Extension now offers integration with Looker. Users on Looker web pages can access Alation metadata directly through the extension for Dashboards, Folders, and Reports, provided these Looker objects are cataloged in Alation. (AL-174521)

• Fixed an issue where logout didn’t work from the Search tab. (AL-175333)

• Fixed an incorrect message on the Snowflake tables list page. (AL-175336)

• Removed the Compose button from BI Report and BI Dashboard pages when opened in the extension. (AL-175334)

Version 1.5

Released October 14, 2024

• The Alation Chrome Extension now supports Power BI. Users with the extension installed in Chrome can view Alation catalog details for reviewed objects directly within the Power BI cloud environment. This integration enhances the visibility of metadata for users navigating Power BI. (AL-167682)

• Updated the Alation logo. (AL-173518)

• Prepared the Alation Chrome Extension for future localization by updating the codebase to support multiple languages. Localization features are coming in a future release. This update lays the groundwork for global user accessibility. (AL-173141)

• Fixed an issue where mentions in the Description fields were shown without the object type icon in the Alation Chrome Extension previews. (AL-172408)

• Removed the Path section from file system object preview. (AL-170798)

• Fixed an issue where titles were not displayed for File systems in the Alation Chrome Extension previews. (AL-168787)

• Fixed an issue where the object type icon wasn’t displayed for API resource objects. (AL-169800)

• Fixed an issue where images added to the Description field weren’t displayed in the Alation Chrome Extension previews. (AL-168777)

• Fixed an issue where names for untitled domains were not visible in the Alation Chrome Extension previews. Now, the placeholder name “Untitled Domain [id]” will be displayed. (AL-166492)

Bug Fixes

• Blocked unauthenticated requests to the Alation Analytics A API endpoint for showing the Top Users Leaderboard. (AL-172067)

• Fixed a validation error that was previously displayed before the user could add new credentials when adding tables to data policies. (Alation Cloud Service on the cloud-native architecture, New User Experience) (AL-152425)

• Users with the Explorer role won’t be able to navigate to Compose. They will be able to use query forms and view catalog pages. (Alation Cloud Service on the cloud-native architecture) (AL-110496)

• Fixed loading times of the custom fields list for document objects. Fetching all custom fields previously impacted performance, especially in environments with a large number of fields. Now, we’ve replaced the dropdown that previously listed all custom fields with Search widget. The list initially displays 100 fields, also allowing users to search and select the desired custom field if it’s not present in the initial list. This change applies to the Update Custom Fields dialog in Bulk Actions, Add Reviewers dialog in Workflows, and the Field Changed Is filter on the Catalog Activity report. (Alation Cloud Service instances on the cloud-native architecture, New User Experience) (AL-174130)

• Fixed an issue with the view definition SQL for Impala data sources where it could not be displayed with an error: This Query cannot be displayed (parsing error). Fixed by the GSP upgrade. (AL-166087)

• Fixed an issue with parsing the queries having the DELETE keyword in SAP HANA data sources. Fixed by the GSP upgrade in version 2024.3.2. (AL-163822)

• Fixed a performance issue with loading Document Hubs after populating an Object Set custom field on their catalog page. Optimized the code for a bulk fetch of permissions instead of fetching permissions for each object individually for Object Set fields. (AL-168264)

• Enhanced logging of access control for Document Hubs and the folder and document objects. Now capturing changes to the Public or Private setting, assigning an access level to users, and updating the permissions inheritance setting. (AL-168606)

• Fixed an issue in Bulk Utility where the migration failed with the error local variable my_tag referenced before assignment. (AL-171021)

• Bulk Utility flags are now read from the database instead of a configuration file. This change ensures settings are preserved even after the pod restarts in the Cloud Service, enhancing reliability. (AL-172541)

• Resolved an issue where documents were not appearing in the left-hand navigation panel. (AL-176260)

• At-mentions found in the body of the article will now be updated during migration via the Bulk Utility. (AL-159340)

• Eliminated an internal API call previously made to fetch the data source version after opening a connection in Compose. This change addresses performance lags observed in certain data sources with connections not going through Query Service. (AL-173363)

• Updated the query log ingestion logic to extract all queries that are part of a SQL transaction. (AL-171721)

• Fixed the issue of the compute_mention_stats job failure due to the identity sequence. (AL-166733)

• Resolved a problem where predicate attributes were incorrectly assigned to multiple tables sharing the same attribute name. Updated the logic to ensure attributes are correctly assigned to the appropriate table. (AL-134202)

• Resolved an issue where navigating to a file object in the file system incorrectly triggered a 404 error (Not Found) from the domains membership API. This fix ensures proper handling and response when accessing file objects. (AL-175777)

• Fixed an issue where the predicate_value_stream internal API returned literal values when literal obfuscation was enabled, and users could read literal values through this API. This fix blocks the endpoint with a 403 response (Forbidden) in case literal obfuscation is enabled. (AL-174585)

• Fixed a bug where users and groups given access to a query were sent notification emails when the option to send notifications to users wasn’t selected. Now, users in the group will be notified only when the checkbox is enabled. (AL-173858)

PATCH RELEASE 2024.3.1.3

build 19.1.3.90272

Released October 18, 2024

Note

This patch release is available for Alation Cloud Service instances on the cloud-native architecture. This patch release doesn’t apply to customer-managed (on-premise) instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

• Fixed an issue where Datadog metrics were not being sent or received across multiple clusters and tenants. The update ensures that metrics and logs are now properly handled. (AL-176268)

PATCH RELEASE 2024.3.1.2

build 19.1.2.89681

Released October 16, 2024

Note

This patch release is available for customer-managed (on-premise) instances only. This patch release doesn’t apply to Alation Cloud Service instances.

• Fixed an issue where users experienced performance degradation during metadata extraction and query log ingestion (QLI), characterized by backed-up task queues, QLI executions missing the schedule, and high CPU usage by Celery processes. The performance issues were traced to a specific method that was called frequently, each time taking 0.1-1 second per call and slowing the system. We implemented optimizations to decrease the frequency and impact of database calls made by this method, enhancing the QLI performance. (AL-174546)

PATCH RELEASE 2024.3.1.1

build 19.1.1.89401

Released October 11, 2024

Note

This patch release is available for Alation Cloud Service instances on the cloud-native architecture. This patch release doesn’t apply to customer-managed (on-premise) instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

• Addressed an issue where Server Admins of Alation Cloud Service instances were receiving excessive email notifications for missed Celery jobs, occurring every 30 minutes. We’ve introduced a feature flag Enable Email Alerts for Celery Jobs under Admin Settings > Feature Configuration to manage such email notifications. This feature flag is disabled by default and can be enabled as needed. When enabled, it triggers hourly and daily checks, notifying admins of critical issues like missed jobs and task queue backlogs in the Celery queues ['default', 'ingestion', 'run_scheduling', 'stewardship']. (AL-174801)

• Fixed an issue where the watchdog process responsible for backing up site data files failed to start during the provisioning of new tenants. (AL-174982 )

RELEASE 2024.3.1

build 19.1.0.88821

Released October 03, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

In this release:

• New User Experience Updates

• Lineage

• Compose

• Search

• Platform

• Improvements

• Bug Fixes

New User Experience Updates

Alation Cloud Service instances on the cloud-native architecture

• Additional customization options for Server Admins:

  • Ability to configure primary and secondary application colors in Admin Settings > Customize Branding

  • Ability to hide the Switch to New Experience/Switch to Classic Experience button using the new feature configuration flag Remove the Back to Classic button in the new UI in Admin Settings > Feature Configuration

• More enhancements to the wayfinder (the left-side navigation panel):

  • Automatically hiding the objects that haven’t been fully configured for non-administrator users to prevent confusion

  • NoSQL objects have been incorporated under the Data Sources hierarchy

  • Stability improvements across several object types

• Miscellaneous other updates:

  • ALLIE-generated titles can be reviewed from the parent object pages

  • Mentions within rich text fields are now easier to trigger

  • Object set field values in the main panel now support pagination

  • The Filters tab now indicates if a filter is required or not

  • The Source Comments field formatting is now in parity with the Classic Experience

  • The data type STRUCT on the column catalog page is now included

  • The About this Instance page now displays the Frontend build number reflecting the current version of the New User Experience

Find more details in New User Experience.

Lineage

Custom Field Metadata Layers (Beta)

Server Admins and Catalog Admins can now configure custom fields to appear as metadata layers in lineage charts. Users can select custom field metadata layers on the Compound Layout lineage chart view from the Metadata Layer list. When applied to the chart, the layer displays the corresponding custom field value for the lineage nodes that have this field. Custom field metadata layers can be created in Admin Settings > Lineage Settings > Custom Field Label. (AL-165274)

Note

The feature is controlled by the alation_conf parameter alation.feature_flags.dev_enable_lineage_custom_field_metadata and is enabled by default.

Bulk-Import Lineage Information (Beta)

The issues with this beta feature that were identified in version 2024.3 have now been addressed:

• The upload template can now be downloaded on Alation Cloud Service instances.

• On newly created instances, there are no more issues with uploading the template.

The feature is enabled by default on the Alation Cloud Service instances. On customer-managed instances, the feature can be enabled by setting the alation_conf parameter alation_conf alation.feature_flags.dev_enable_lineage_bulk_upload to true.

Lineage between RDBMS, BI, and file system objects can be uploaded using the template. External objects are currently not supported. The row limit for uploading is 200.

Deprecation Warning Propagation Worker Functionality

Alation Cloud Service instances on the cloud-native architecture

We’ve refactored the legacy deprecation warning propagation service for lineage actions into a new service. This update enhances the handling of deprecation-specific propagation events. Key improvements include:

• Automatic adjustment of deprecation propagation when view definitions change. If a view definition changes for a particular downstream object that was along the deprecation path from an upstream object, then deprecation propagation will adjust to not highlight the view.

• Consistent addition and removal of propagated deprecation warnings across table, column, data source, and schema-level objects.

• Dynamic updates to the lineage chart when dataflows are added or removed, reflecting accurate deprecation warning paths. (AL-154463)

Compose

Queuing in Compose with Query Service

For data sources where Query Service is enabled, we’ve implemented a concurrency control queue for query executions done within the same connection. This ensures that for a particular connection, we can queue multiple query executions without the Concurrent queries cannot be executed error. Currently, this feature is disabled by default, and will be enabled by default in a later release. To activate it now, append the JVM flag -Dconnector.use_concurrent_execution_per_connection=true preceded by a space to the existing values of the parameter connector.extra_flags in alation_conf. Previously, users would encounter an error in the results pane when attempting to run multiple queries on the same connection while another was still processing. With the new concurrency control, users can submit several queries simultaneously. These queries will be queued and executed serially in the background, similar to how queries are managed when exceeding the maximum concurrent query limit.(AL-166099)

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

Search

Exact Search Without Double Quotes

For search strings without double quotes, whether they contain underscores or not, titles or names that exactly match the search query will now appear at the top of the search results. Previously, search strings with underscores were broken into tokens and ranked based on the scores of each token, which did not prioritize exact matches. (AL-171164)

Platform

PostgreSQL Upgrade to 16.4

Customer-managed instances

The internal PostgreSQL databases (the internal Alation database and the Alation Analytics database) have been upgraded from version 16.2 to 16.4 to enhance application security. The databases will be upgraded automatically during the Alation update to 2024.3.1. (AL-169856)

Ability to Enable SSL for the Kafka and Zookeeper Components

Customer-managed instances

Alation now supports enabling SSL/TLS encryption for Kafka and SASL over TLS authentication for Kafka clients and Zookeeper. Even though all services, producers, and consumers run on the Alation server and use localhost for connections, this enhancement addresses potential security concerns. (AL-122489)

Note

The feature is disabled by default and can be enabled by the instance administrator on the server backend. Steps to enable:

sudo su alation
rm -rf /var/lib/event_bus/kafka-ssl/*
alation_conf event_bus.kafka.ssl.enabled -s True
alation_action kafka-generate-ssl
alation_action deploy_conf_kafka_ssl
alation_supervisor restart event-bus:*
alation_supervisor restart lineage

Java Component Upgrade

The internal Java component has been upgraded to a newer version (Corretto 11.0.24.x.x) to incorporate the latest security fixes and enhancements. (AL-164803)

Enhanced Audit Logging for Database Connectors, File System Connectors, and Custom Field Permissions

The user actions on the data source settings page are now audited and the corresponding logs are stored in alation-event.log in the /opt/alation/site/logs directory. The log captures information about the user who triggered the action, when it was triggered, what was the outcome of the action, what was the previous value of the field before the user acted on it, and what is the new value changed by the user. For example, if a user changes the URI for a source, this action will be captured by the log. (AL-123286, AL-168788, AL-168620)

Improvements

• Optimized the lineage compound layout to eliminate redundant calculations. This enhancement will significantly improve chart load times. (AL-170941)

• Implemented support for non-conforming session IDs in the QLI pipeline service, which now strips extraneous details such as usernames and timestamps from the session ID in custom MDE queries. This correction was implemented for the Snowflake OCF connector and will be extended to all other OCF connectors in the future. (AL-168154, Alation Cloud Service instances on the cloud-native architecture)

• The alation_conf flag enable_obfuscating_view_sql is now set to true by default. This ensures that when the Obfuscate Literals option is enabled for a data source, literals in the extracted view SQL will be automatically obfuscated. (AL-163780)

• Updated the scan_postgres utility:

  • Integrated scan_postgres with pg_amcheck for improved performance and reliability

  • Added progress indicators

  • Enhanced efficiency for larger databases (AL-164467, Customer-managed instances)

• Previously, some customers experienced congestion in Celery Worker resources due to a large number of catalog set-related tasks getting queued. We’ve implemented a rate limit on the affected tasks to deduplicate tasks with identical arguments (such as the catalog set ID). Due to rate limiting, only one unique task will be queued and executed within a configured interval. This approach ensures more efficient use of resources and reduces task backlog. Rate limiting was added for the sync_members_from_catalog_set and update_dynamic_sets tasks by debouncing them within a specified interval. The interval is controlled by the alation_conf flags stewardship.sync_members_from_catalog_set.debounce_interval and stewardship.update_dynamic_sets.debounce_interval respectively, with a default interval of 600 seconds. During the debounce period, updates to catalog set members will not reflect in these features if the task is rescheduled within this time.

  Impact on features:

  • The sync_members_from_catalog_set debounce period: Delays the synchronization for the Catalog Activity and Mentioned On features for Documents and Folders by the debounce interval.

  • The update_dynamic_sets debounce period: Delays the synchronization for Catalog Sets Membership and updates to Catalog Sets Shared Fields by the debounce interval. (AL-171500)

Bug Fixes

• Fixed an issue where the bulk upload of lineage with an Excel template failed on newly installed instances. (AL-172593)

• Fixed an issue where objects that were @-mentioned in rich text fields displayed only icons, without titles. After this fix, the titles of referenced objects are displayed as expected. (AL-172065)

• Fixed an issue where characters like the ampersand & appeared in their HTML-escaped form in catalog set titles on catalog pages. (AL-128763)

• Previously, any query execution error incorrectly included an additional message: If unable to reconnect, please validate if your Hive Server is up and running properly. This issue has been resolved. Now, only the syntax error message relevant to the current database type is displayed. (AL-169746)

• Alation Cloud Service instances on the cloud-native architecture: Fixed a bug where published queries from Compose were sometimes not shown in the catalog under the table catalog page > Queries. (AL-152405)

• Fixed an issue where some data dictionary-related tasks were stuck in processing or queued states due to concurrent invocations. To address this, concurrent API requests on the same resource will now be blocked and return a 400 error status, preventing tasks from getting stuck. Additionally, a daily scheduled task has been implemented to scan the database for any data dictionary tasks running for over 24 hours. The expiration threshold can be adjusted with the alation_conf flag alation.data_dictionary.task.pending.seconds_to_keep. (AL-168954)

• Alation Cloud Service instances on the cloud-native architecture: In the New User Experience, fixed an issue where a generic error message was shown when a data dictionary export was already in progress. Implemented status code checks to improve error messaging. (AL-168794)

• Alation Cloud Service instances on the cloud-native architecture: Fixed a pagination issue in the New User Experience, ensuring that object set values within template main sections now match the pagination behavior of the Classic Experience. (AL-165421)

• Customer-managed instances: Fixed an issue where the health check monitor URL (<Base_URL>/monitor/health_check/) previously took about 2 minutes to display results. Health checks now load in less than 3 seconds on average. (AL-168548)

• Fixed an issue with the Kafka processes using plain text listeners. We’ve introduced the ability to enable encryption for both Kafka and Zookeeper processes, enhancing security. (AL-143490)

• Fixed an issue with the Snowflake OCF connector with the dbt add-on connector where metadata extraction (MDE) failed with the error Unrecognized token ‘version’: was expecting (JSON String, Number, Array, Object or token ‘null’, ‘true’ or ‘false’) when the manifest file was large (>100MB). Added a fix to get large manifest files from the correct raw GitHub URL. (AL-167304)

• Fixed a bug where using the public API to add or remove domain memberships (integration/v2/domain/membership/) with 100 or more objects recursively resulted in a 500 error and a code exception. Updated the API exception handling logic to include the limit argument, ensuring that a correct 400 response with a detailed message is returned. (AL-165401)

• Fixed an issue where max(last_processed_record_ts) for bi_report and documenthub in Alation Analytics showed a future date due to an ETL bug that incorrectly set the timestamp when the ETL checkpoint was reset. This fix ensures accurate timestamp setting during the ETL process. (AL-164137)

• Fixed an issue where a user who was provided access to the Alation Analytics data source, was suspended before they could create their password from the Account Settings > DB Connections tab. With this fix, only those users whose credentials already exist can be suspended. (AL-162736)

• Fixed an issue where the ts_updated value for custom fields wasn’t reflected in the Alation Analytics database. Now, the ts_updated field for all the object types will be updated in case a custom field associated with an object is updated in the catalog. (AL-161824)

• Fixed an issue where Alation Analytics wasn’t capturing deleted fields for data sources, schemas, tables, and columns. Updated the query used by the Analytics ETL to ensure we capture records which have been deleted. (AL-155423)

• Fixed the Analytics ETL to ensure ts_deleted is reflected correctly for Policy tables. (AL-170039)

• Fixed a bug in the loading of the curation_history data into Alation Analytics, which caused some records to be missed. (AL-165866)

PATCH RELEASE 2024.3.0.1

build 19.0.1.87345

Released September 20, 2024

Note

This patch release is available for Alation Cloud Service instances on the cloud-native architecture. This patch release doesn’t apply to customer-managed (on-premise) instances and Alation Cloud Service instances that haven’t been migrated to the cloud-native architecture.

• Fixed an issue where objects that were @-mentioned in rich text fields displayed only icons, without titles. After this fix, the titles of referenced objects are displayed as expected. (AL-172065)

RELEASE 2024.3.0

build 19.0.0.85267

Released September 3, 2024

Note

This release is available for all Alation instance types: customer-managed (on-premise) and Alation Cloud Service.

In this release:

• New User Experience Updates

• Alation Anywhere

• Homepage

• Alation Analytics

• Compose

• Lineage

• Search

• Authentication and User Management

• Governance

• Open Connector Framework

• Bug Fixes

• Known Issues

New User Experience Updates

Alation Cloud Service instances on the cloud-native architecture

Ability to Remove the Switch to New Experience/Switch to Classic Experience Button

We’ve introduced a new feature flag, Remove the Back to Classic button in the new UI, which allows Server Admins to hide the Switch to New Experience/Switch to Classic Experience button from the user interface. Enabling this feature establishes the New User Experience as the user interface without the ability to go back to the Classic Experience. Pages that haven’t yet been converted to the New Experience will still be presented in the Classic Experience. (AL-164824)

Alation Anywhere

Alation Cloud Service instances on the cloud-native architecture

Public Preview of Alation Chrome Extension (Beta)

The Alation Chrome Extension enhances browser functionality by allowing Alation users to search and preview data catalog objects directly in Chrome. After installing the extension from the Chrome Web Store, users can log into their Alation cloud instance, conduct quick searches, and view metadata previews. Snowflake users get additional benefits from this integration. After signing into Snowflake Snowsight, users can access detailed metadata about schemas and tables directly through the extension, enriching their data context with metadata from Alation, without leaving Snowflake. (AL-159629, AL-159628)

Homepage

Dynamically Rearrange Custom Blocks on the Homepage

Alation Cloud Service instances on the cloud-native architecture

We’ve introduced a dynamic user interface enhancement, enabling the flexible positioning of custom blocks on the homepage. Server Admins and Catalog Admins can now rearrange homepage blocks by dragging-and-dropping them. Save the changes for the updates to be applied to the homepage. (AL-153691)

Alation Analytics

Group Access Control for Alation Analytics

Data Source Admins now have the capability to manage access to the Alation Analytics data source through groups. On the Access tab of the Alation Analytics data source settings, you can add or remove entire groups and adjust their access levels. Previously, access had to be granted to users individually, which could become inefficient with the increasing number of Alation Analytics users. (AL-82666)

Compose

Scheduled Query Execution Notifications for Allowed Users and Groups

Alation now sends notifications on scheduled query execution to the users and groups with whom the results have been previously shared. To opt out, users can disable all scheduled query alerts via their profile preferences. The feature is controlled with two alation_conf flags:

• notify_allowed_users_and_groups: Enables or disables sending notifications to users and/or groups listed in the Scheduled Query Setting > Sharing Settings dialog, with whom the results have been shared previously. Enabled by default.

• notify_users_in_groups: Enables notifications directly to individual users in a group specified in the Scheduled Query Settings> Sharing Settings dialog. Disabled by default, and the specified group is notified through the group email. (AL-168820)

The Sapphire Compose Theme

The Sapphire Compose theme is designed to complement the New User Experience. It will be automatically applied for all users after an admin enables the New User Experience feature flag. This ensures a consistent aesthetic across all Alation interfaces. If users opt to switch from the Sapphire theme to another, the auto-application of the theme will be disabled, and subsequent themes will require manual selection. (AL-159192)

Note

The Sapphire theme is also available on customer-managed (on-premise) Alation instances.

Differentiate Between Launching and Running a Query

We’ve introduced a new status indicator for running queries to differentiate between launching and running a query. Previously, the status message Launching query was displayed for the duration of the query execution. Now, it’ll change to Executing query once the database connection is established. This enhancement is intended to help discern the stage of query processing and quickly spot any connection delays. (AL-21736)

Change to Default Setting for Automatic Connections

The default value for the alation_conf parameter allow_automatic_connection_on_tab_open has been changed to false. This update disables the automatic creation of connections to the datasource when users open a new tab or switch tabs in Compose, causing database account lockout problems when passwords have expired. (AL-165067)

Change to Default Setting for DDL Query Processing

We’ve updated the default behavior for processing of Data Definition Language (DDL) queries from Compose. Previously, the incremental extraction and lineage generation for DDL queries ran synchronously. Now, the alation_conf parameter alation.query_exec.ddl.run_callbacks_async setting is enabled by default, and these processes will run asynchronously. This update aims to enhance performance and reduce the time it takes for queries to complete. (AL-136855)

Deprecation of Desktop Compose

The Desktop Compose installer is no longer available for download from Alation, and the option to select Desktop or Web as the default Compose user experience has been removed from admin and user settings. Existing Desktop Compose installations will continue to function, but users are encouraged to transition to the Compose Web interface that is accessible via a regular browser. All queries and settings like theme and font size are preserved in the Web interface, but open tabs and connections will need to be re-established. Critical support for Desktop Compose will cease by August 2025. (AL-154706)

Lineage

Lineage Support for SQL Server Stored Procedures

Alation Cloud Service instances on the cloud-native architecture

We’ve enhanced lineage support for cataloged stored procedures and functions for SQL Server OCF connectors. It is available for systems on the cloud-native architecture and the mt_lineage service enabled. To activate this feature, the alation_conf parameter alation.ocf.mde.rdbms.parse_function_ddl.enabled should be set to true. This enables the detailed parsing and lineage tracing of function and procedure definitions. (AL-156708)

Known Issue with SQL Server Lineage from Stored Procedures

The lineage chart may not be accurate when queries include columns with unqualified names. (AL-170059, AL-170264)

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

Lineage Minimap Available by Default

Alation Cloud Service instances on the cloud-native architecture

The lineage minimap feature in lineage charts no longer requires enablement through a feature flag and is available by default. (AL-168144)

Bulk-Import Lineage Information (Beta)

Users with the Data Stewards, Data Source Admins, Catalog Admins, and Server Admins roles can now bulk-import lineage information via an Excel template. This feature allows users to easily identify the success or failure of each submission through a detailed upload report. The feature is currently in beta and can be enabled with the alation_conf parameter alation.feature_flags.dev_enable_lineage_bulk_upload. (AL-159481)

Note

UPDATE

September 20, 2024

We have identified a few issues affecting the functionality of this beta feature in version 2024.3:

• The Excel template cannot currently be downloaded on Alation Cloud Service instances.

• On newly created instances, there are issues with uploading the template.

The Engineering team is actively working on resolving these issues to ensure full functionality in the next version, 2024.3.1, planned for early October 2024. In the meantime, we advise refraining from using this beta feature. For any concerns or immediate needs, please contact the Alation Support team.

Lineage Settings User Interface for RDBMS OCF Connectors (Beta)

Data Source Admins can now use a separate user interface for configuring cross-system lineage. The feature is currently in beta and can be enabled with the alation_conf parameter alation.feature_flags.DEV_enable_etl_elt_lineage_settings. (AL-152599)

Note

Alation Cloud Service customers can request server configuration changes through Alation Support.

Search

Struct Column Visibility in Search

For data sources containing STRUCT data types, the visibility of STRUCT columns can now be managed via Per-Object Parameter settings for schemas and tables. When a schema or table is configured as non-browsable, all STRUCT columns within that object will automatically be hidden. This ensures that these columns are no longer searchable in Alation when their parent objects are set to non-browsable, providing better access control. (AL-152087)

Authentication and User Management

User Profile Upload Limited to 25 Updates

We’ve introduced new limits for uploading user profiles. Each CSV can now contain no more than 25 user creations or updates. If a CSV contains more than 25 rows but the total number of user creations and updates does not exceed 25, the upload will proceed. Details about this restriction are available in the info panel in Admin Settings > User Profiles. Uploads exceeding this limit will show a preview but disable the Confirm button, with an error message displayed below the preview area. (AL-166371)

Bypassing SAML Authentication Is Disabled

The mechanism to bypass SAML authentication has been disabled for administrators of both Alation Cloud Service on the cloud-native architecture and on-premise deployments. In emergency situations, instance admins will need to contact Alation Support to request a temporary bypass. (AL-127742)

Governance

Improvements to Domain Membership Rules and Performance

Catalog Pages will now always accurately reflect the object’s Domain membership, without the lag time that was previously seen. The rules for domain propagation have changed. Previously, exclusions of child objects from Domains were very persistent:

• An exclusion rule was permanent unless an object was added to the Domain directly.

• When a parent object was included recursively, then child objects were only added to the Domain if no parent objects at any level were excluded.

Now, the most recent action takes precedence. For example, if a user adds a Schema recursively this week, it will override a previous Domain exclusion on a child Table from last month. This change is designed to align more closely with user expectations, ensuring that catalog pages immediately and accurately reflect an object’s Domain membership. (AL-166059, AL-166060)

Handling of People Set Reviewer Deletion in Workflows

We’ve updated how existing workflows handle the removal of People Sets designated as reviewers. If a People Set set as a Reviewer is removed from the system, it will now be marked as [Gone] in the Reviewer field to indicate its absence. (AL-164891)

De-duplication of Snowflake Tags Enabled by Default

The alation_conf parameter alation.feature_flags.enable_centralized_tags_deduplication is now enabled by default. Snowflake tags from a common database or schema that are used across other databases or schemas will now be de-duplicated by default during extraction. (AL-166048)

Deprecation of the Suggested Stewards Feature

The Steward Suggestions feature is being deprecated. While customer instances with this feature currently enabled can still view, approve, or reject steward suggestions, the option to enable it through the feature flag in Admin Settings > Feature Configuration has been removed. However, the corresponding alation_conf parameter (alation.feature_flags.enable_allie_steward_suggestions) remains available, allowing for the feature to be disabled if necessary. (AL-168008)

Open Connector Framework

OCF Connectors

Updates to the Open Connector Framework (OCF) connectors are released every week. See OCF Connector Release Notes for information on the updates in the month of August 2024. A few August highlights are:

Databricks Unity Catalog OCF Connector Improvements

We have released version 3.0.0 of the Databricks Unity Catalog OCF connector that enhances its metadata extraction capabilities. This update includes switching from a driver-based extraction method to a more robust query-based extraction from the information_schema. (AL-157004)

Enable Token Refreshing for Google BigQuery Compose OAuth

We’ve added the capability to auto-refresh OAuth tokens via the Compose authentication flow for Google BigQuery data sources. Now, when Data Source Admins enable the Request Refresh Tokens checkbox on the Compose tab of the settings page, Alation will auto-refresh tokens for connections established in Compose. (AL-162370)

Improved Workspace Extraction from Azure Power BI

Version 2.4.0 of the Power BI Scanner connector introduces an option to Extract all workspaces from Azure Power BI within the Customize additional extraction scope section on the Metadata Extraction tab.

• Default Behavior: With the Extract all workspaces from Azure Power BI toggle turned off, only workspaces accessible to the configured service principal will be extracted.

• Enhanced Extraction: When the toggle is turned on, all workspaces from Azure Power BI will be extracted, regardless of the service principal’s access permissions.

Additionally, this version supports lineage from Redshift native query-based datasets and Snowflake-based datasets. (AL-168903)

Bug Fixes

New User Experience

Alation Cloud Service instances on the cloud-native architecture

• Previously, if a policy title consisted only of multiple spaces, it would display as blank and would be non-editable. Now, titles composed of multiple spaces will be displayed as Untitled <object type>. (AL-163248)

• In the Policy Center, the policy group membership dialog previously showed the Save button enabled by default, even if no groups were selected. We’ve updated this behavior so that the Save button will only become enabled when a user selects or deselects any policy group. (AL-162835)

• Previously, when the Enable New UI - Default and UI Promotion access flag was enabled, links on some pages under Govern would still redirect to pages on the Classic Experience. Now, all pages will correctly redirect to New User Experience when it is set as the default behavior. (AL-163421)

• Fixed an issue with the Title field validation when creating new workflows. (AL-163548)

• Addressed an issue that occurred when attempting to edit workflows associated with custom fields that had been deleted from the system. Previously, trying to edit such workflows would result in an error in both Classic and Neo Experiences. Now, instead of blocking the user, the system will display the label [Gone] for the deleted custom field within the workflow. (AL-166130)

• Previously, when users were assigning Reviewers to workflows, the Reviewer list failed to display all existing People Sets. This fix ensures that all People Sets are now visible when adding a Reviewer to workflows. Additionally, this issue was fixed in the bulk actions and catalog activity features, ensuring consistent functionality. (AL-166710)

• Addressed an issue where changes to the order of columns in the Document table were not persisting after users attempted to swap them and then save. (AL-163769)

• Fixed an issue where a homepage name disappeared from the list of homepages in Admin Settings > Homepage Customization when the homepage was disabled and then enabled and saved or enabled and then disabled enabled and saved. Now, the homepage name remains visible in the list after enabling or disabling. (AL-167711)

• Fixed an issue with adding terms to custom blocks on the homepage where adding a term, folder, or document to a custom block on any homepage resulted in an unexpected error. These objects can now be successfully added to custom blocks without errors. (AL-167993)

• Fixed an issue where logo updates done on the Admin Settings > Homepage Customization page weren’t reflected on the homepage despite a message indicating a successful save. The issue has been resolved, and the logo now updates correctly in the top left corner after saving. (AL-168163)

• Fixed an issue where the Column Organization widget was missing from the Policy Center in New User Experience. The Column Organization feature is now available for policy pages. (AL-162838)

OCF Platform and Connectors

• Fixed a bug in the OCF downstream job for tags that previously led to errors when handling missing tags. The issue caused deleted tag links in the source to remain uncleared in Alation. With this fix, the system now correctly cleans up missing tag links. (AL-166880)

• Enhanced the ingestion process for Snowflake tags to address an issue where tag values containing newline characters were not being handled properly. This fix ensures that all options for the Snowflake tag picker field are populated correctly. (AL-164875)

• Updated the SQL Parser grammar type for the Athena OCF connector to address issues with query parsing. This enhancement improves the accuracy of parsing and ingesting queries from Athena into Alation. (AL-167009)

• Resolved an issue where the GET_DDL() function was called indiscriminately for procedures and functions in Snowflake, leading to extraction query failures and multiple errors. This update enhances the stability and efficiency of metadata extraction from Snowflake data sources. (AL-163439)

• Fixed an issue in the Snowflake OCF connector data sampling process. Previously, when the maximum rows to store was set to 0, the table row count was incorrectly displayed as 0, rather than reflecting the actual row count. This has been corrected in connector version 2.4.4 to ensure that the row count accurately represents the data in the table, regardless of the max rows setting. (AL-162276)

• Resolved a limitation in the Tableau OCF connector where Excel sheets embedded from OneDrive weren’t appearing on the Connections tab in Alation. We have now added support for the creation of connections for cloud file types with connector version 1.9.5. (AL-161089)

• Fixed an issue with storing STRUCT columns in the Alation internal database as NoSQL attribute objects where some of them were duplicated as could be discovered through Search. Duplicating is the result of the internal logical model for NoSQL data types. After this fix, duplicate objects will no longer be discoverable by users, as they only service the internal model. (AL-147379)

• Fixed an issue where the impact analysis report for a Snowflake data source resulted in empty Path values for BI objects. With this fix, the correct path information will be displayed for BI objects in both Impact Analysis and Upstream Audit reports, ensuring lineage traceability. (AL-164388)

• Resolved an issue after upgrading to version 2023.3.5.1, where the connector.log file was missing and older connector log files were not being compressed using gzip. This was due to an internal process incorrectly removing the connector from the log path. A conditional check has now been implemented to prevent this from happening. (AL-161279)

Governance

• Resolved an issue in the Stewardship Workbench Bulk Actions feature, where the Select All option incorrectly applied actions to more objects than displayed in the search results. Now, when using the Select All option, only the objects shown in the search results will be updated. (AL-163762)

• Optimized the internal APIs behind the workflow step execution after observing slow response times in logs. This enhancement reduces the overall execution time of the APIs. (AL-161158)

• Corrected a typo in the spelling of Catalog Activity on the Curate and Govern landing page in the Classic Experience. (AL-164324)

• Resolved an issue that prevented users from editing existing workflows, which was caused by the system attempting to fetch all users and groups. The solution now bypasses the user or group fetch API if no unique users or group profiles are added as reviewers. This change eliminates unnecessary data retrieval, ensuring that workflow edits can be made efficiently. (AL-164472)

• Fixed an issue causing data dictionary uploads to fail with a 500 internal server error. The problem was traced back to extra double quote characters that were introduced by a third-party CSV editor tool. The fix handles these characters properly, ensuring successful uploads. (AL-165445)

• Resolved an issue in the Bulk Utility where uploading XLSX files with terms resulted in a 504 time-out error. (AL-161711)

• We’ve introduced the capability to schedule a backend job that recomputes the members of all conditional catalog sets, which is useful for addressing an issue where previously deleted objects that were restored via an MDE job, did not appear in conditional sets with matching rules. This job is disabled by default and can be enabled with the feature flag stewardship.DEV_recompute_conditional_catalog_sets_members.enabled. You can customize the schedule by modifying the following feature flags according to Celery crontab standards:

  • Day of week: stewardship.DEV_recompute_conditional_catalog_sets_members.schedule.day

  • Hour: stewardship.DEV_recompute_conditional_catalog_sets_members.schedule.hour

  • Minute: stewardship.DEV_recompute_conditional_catalog_sets_members.schedule.minute

  Changes to these scheduling flags also require a restart of the celery-beat service to take effect. We recommend contacting Alation Support if you observe this issue and need to adjust the flags. (AL-168175)

Alation Analytics and Analytics Experience

• Resolved a synchronization issue between the catalog and the Alation Analytics database regarding deleted fields. Although fields marked as deleted in Rosemeta were accurately updated, this change was not reflected in Alation Analytics. This was corrected by updating the ETL query to ensure it now properly includes records marked as deleted. (Alation Cloud Service instances on the cloud-native architecture, AL-155423)

• Fixed an issue where Steward updates in term objects made through Bulk Actions were not appearing in Alation Analytics. The solution includes a migration script in the upgrade package that resets the ETL checkpoint for terms. Consequently, the ETL process for terms will restart from the beginning after the update. Given Alation Analytics’ ETL capacity to process only six months of data per ETL run and depending on the volume of terms data, multiple ETL cycles may be necessary to fully update the terms data. This could result in a delay in reflecting the latest terms changes. (AL-158292)

• Fixed an issue in Alation Analytics that caused errors during data insertion into Snowflake when non-ASCII characters were present in custom fields. (Alation Cloud Service instances on the cloud-native architecture, Analytics Experience, AL-147037)

Other Fixes

• Resolved an issue where Search results filters failed to populate additional filter data when any documentation object type was used as the object filter. Additionally, deleted folders will no longer appear in full search results. (AL-160582)

• Addressed an issue in the LMSTitleDescriptionSerializer, where each call to a specific method triggered a separate query execution. This problem was present when the Annotation Value Service was enabled. To optimize performance, unnecessary calls within the LMSTitleDescriptionSerializer have now been removed. (Alation Cloud Service instances on the cloud-native architecture, AL-164402)

• To enhance application security we have now restricted the uploading of SVG files for articles. (AL-135045)

• Fixed an issue with sending alerts to admin users. The system now performs status checks on Celery jobs every 30 minutes. If a job missed its scheduled run or if any Celery queues become excessively backed up, an email notification will be sent out. (Alation Cloud Service instances on the cloud-native architecture, AL-156446)

• Fixed an issue where emails were not being sent to group emails when posts were made in conversations. This update ensures that all conversation assignees, including groups, now receive email notifications. (AL-159349)

• Previously, users had no option to return to the original object page from the data dictionary upload page except by using the browser’s Back button. Now, users can easily navigate back to the data source from the upload dictionary page using a breadcrumb trail. (AL-152150)

• Added support for displaying partition key information in the table view of Alation Search. The fix applies to any data source that supports partition keys and includes partition key information in its extracted metadata. (AL-159752)

Known Issues

• Alation Cloud Service instances on the cloud-native architecture: In version 2024.3, objects that are @-mentioned in rich text fields display only icons, without titles. Such objects as articles, terms, documents, and domains are affected. Please note, the titles are intact in the internal database but are not appearing in the user interface. A fix has been provided with PATCH RELEASE 2024.3.0.1. (AL-172065)

• Conversations that are deleted and subsequently restored are currently not appearing in the Inbox. (AL-169183)