Load runtime configuration from environment vars, configuration file, or command line #187
Description
Is your feature request related to a problem or challenge? Please describe what you are trying to do.
A Physical plan that is executed has several parameters that would be good to have a way to control via configuration variables. We should have a mechanism, sort of like Ballista, where users can set Environment Variables, load a TOML configuration file, or pass in parameters to function calls via a Python dict with key/value pairs.
Describe the solution you'd like
- Reasonable default values for configurations
- File based configurations possible for overrides
- Functions that trigger "runs" to have a Python Dict that allows for override of global configuration values.
Describe alternatives you've considered
None
Additional context
Add any other context or screenshots about the feature request here.
Activity
timsaucer commentedon Apr 9, 2026
I believe this is now fully supported in the current interface between the built in config parameters we can set in python plus the recent addition of custom configuration via #1391
Please feel free to reopen if there are additional means you would like to set these configs. I see you mentioned file based config. We haven't received additional comments or requests for this and based on the age of the issue it feels like it may be low priority. If you think this is essential, please reopen the issue and we'll address it. I don't think adding something like toml support is a very hard ask, we just need to know what the desired workflow is.