-
Notifications
You must be signed in to change notification settings - Fork 743
Open
Description
It would be nice to have an interface that converts a SedonaSpark DataFrame to a SedonaDB DataFrame easily. Here is a current solution that works:
import sedona.db
sd = sedona.db.connect()
df = sd.create_data_frame(dataframe_to_arrow(spark_df))This could be nice:
spark_df.to_sedonadb()But maybe we'd have to do this:
spark_df.to_sedonadb(sd)This would allow for cool spatial workflows, like this:
- Read an Iceberg table with SedonaSpark and perform big data operations with a filtering operation at the end to make the data small enough to fit on a single machine
- Convert the SedonaSpark DataFrame to SedonaDB
- Use a library that's compatible with SedonaDB, like lonboard, to create a graph
Let me know what you think!
Metadata
Metadata
Assignees
Labels
No labels