python - How do I GET all Hive Columns with Classifications using the Apache Atlas API?

Question

I am working to sync tags between a Data Catalog software and Apache Atlas. Atlas's version of a tag is a 'classification', so my initial plan is to generate a list of Atlas classifications, compare it to a list of tag associations from the Data Catalog, and reconcile the differences.

I'm looking for the best way to create a result list of all hive columns that have at least one classification, but there doesn't seem to be an API v2 endpoint dedicated to such an operation. API V2 Docs

The best way I've found so far is to:

1. Use the dsl search endpoint to paginate through all entities with type 'hive_column' in chunks of 10K (configured result limit on my Atlas instance)
2. Add relevant metadata from each hive column that has values in its list of 'classifications' to an output list for eventual comparison

This solution gets the job done, but it is not very efficient over 200K hive columns so I'm curious to see whether anyone with Apache Atlas experience has discovered a better solution for targeting classified/tagged columns or other entities.

Answer 1

等待大神答复