steffo/cfig
steffo/cfig
1
Fork 0
mirror of https://github.com/Steffo99/cfig.git synced 2024-11-21 07:24:21 +00:00
Code Activity

💥 Yay more changes

Browse source
This commit is contained in:
Steffo 2022-04-20 04:15:10 +02:00
parent ebb649eb4f
commit 00424a444f
Signed by: steffo
GPG key ID: 6965406171929D01
18 changed files with 328 additions and 282 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

2
.idea/runConfigurations/Sample.xml
View file

@ -13,7 +13,7 @@
<option name="IS_MODULE_SDK" value="true" />
<option name="ADD_CONTENT_ROOTS" value="true" />
<option name="ADD_SOURCE_ROOTS" value="true" />
<option name="SCRIPT_NAME" value="cfig.sample" />
<option name="SCRIPT_NAME" value="cfig.sample.definition" />
<option name="PARAMETERS" value="" />
<option name="SHOW_COMMAND_LINE" value="false" />
<option name="EMULATE_TERMINAL" value="true" />

10
README.md
View file

@ -2,9 +2,7 @@
A configuration manager for Python
\[ [**Documentation**](https://cfig.readthedocs.io/) | [**PyPI**](https://pypi.org/project/cfig/) \]
## Example
\[ [**Example**](https://github.com/Steffo99/cfig/tree/main/cfig/sample) | [**Documentation**](https://cfig.readthedocs.io/) | [**PyPI**](https://pypi.org/project/cfig/) \]
```python
import cfig
@ -13,7 +11,7 @@ config = cfig.Configuration()
@config.required()
def SECRET_KEY(val: str) -> str:
"""Secret string used to manage tokens."""
"""Secret string used to manage HTTP session tokens."""
return val
if __name__ == "__main__":
@ -28,8 +26,10 @@ print(f"My SECRET_KEY is: {SECRET_KEY}")
```console
$ python -m mypackage.mycfig
=== Configuration ===
===== Configuration =====
SECRET_KEY → Required, but not set.
Secret string used to manage HTTP session tokens.
===== End =====
```

1
cfig.iml
View file

@ -12,6 +12,7 @@
<excludeFolder url="file://$MODULE_DIR$/.venv" />
<excludeFolder url="file://$MODULE_DIR$/.pytest_cache" />
<excludeFolder url="file://$MODULE_DIR$/cfig/.pytest_cache" />
<excludeFolder url="file://$MODULE_DIR$/dist" />
</content>
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />

110
cfig/__init__.py
View file

@ -1,113 +1,3 @@
"""
This package provides a simple but powerful configuration manager for Python applications.
A goal is to allow easy integration of an application with multiple configuration standards, such as environment
variables, dotenv files, and Docker Secrets files.
.. code-block:: python
@config.required()
def SECRET_KEY(val: str) -> str:
"Secret string used to manage tokens."
return val
Another goal is to provide informative error messages to the user who is configuring the application, so that they may
understand what they are doing wrong and fix it immediately.
.. code-block:: console
$ python -m cfig.sample
=== Configuration ===
SECRET_KEY → Required, but not set.
Secret string used to manage HTTP session tokens.
HIDDEN_STRING = 'amogus'
A string which may be provided to silently print a string to the console.
Example
=======
Ideally, a "config" module should be created, where the programmer defines the possible configuration options of their
application::
# Import the cfig library
import cfig
# Create a "Configuration" object
config = cfig.Configuration()
# Define configurable values by wrapping functions with the config decorators
# Function name is used by default as the key of the variable to read
@config.required()
def SECRET_KEY(val: str) -> str:
"Secret string used to manage tokens."
return val
@config.required()
def ALLOWED_USERS(val: str) -> int:
"The maximum number of allowed users in the application."
# Values can be processed inside these functions
return int(val)
@config.optional()
def ACCEPTED_TERMS_AND_CONDITIONS(val: Optional[str]) -> bool:
"To accept T&C, set this to a non-blank string."
return val is not None
# If heavy processing is done inside the function, it may be useful to define the configuration key manually
@config.required(key="DATABASE_URI", doc="The URI of the database to be used.")
def DATABASE_ENGINE(val: str):
return sqlalchemy.create_engine(val)
if __name__ == "__main__":
# If the configuration file is executed as main, handle the call and display a user-friendly CLI interface.
config.cli()
Values can later be accessed by the program by importing the configuration file:
.. code-block:: python
# Import the previously defined file
from . import myconfig
# Function is executed once when the value is first accessed
print(f"Maximum allowed users: {myconfig.ALLOWED_USERS}")
# Advanced objects can be loaded directly from the config
Session = sessionmaker(bind=myconfig.DATABASE_ENGINE)
Terminology
===========
In this documentation, the following terms are used:
Key
The name of a configuration value, usually in SCREAMING_SNAKE_CASE.
For example, ``PATH``, the name of the environment variable.
Value
A single non-processed configuration value in :class:`str` form.
For example, the raw string value of an environment variable.
Source
A possible origin of configuration values, such as the environment, or a file.
Proxy
An object used to lazily and transparently resolve and cache values.
After resolving a value, it behaves in almost completely the same way as the object it cached.
Resolver
A function taking in input a value originating from a source, and emitting in output its processed representation.
For example, a resolver may be the :class:`int` class, which converts the value into an integer.
Configuration
A collection of proxies.
"""
# noinspection PyUnresolvedReferences
from .config import *
# noinspection PyUnresolvedReferences

86
cfig/config.py
View file

@ -102,44 +102,7 @@ class Configuration:
log.debug("Initialized successfully!")
def required(self, key: t.Optional[str] = None, doc: t.Optional[str] = None) -> t.Callable[[ct.ResolverRequired], ct.TYPE]:
"""
Mark a function as a resolver for a required configuration value.
It is a decorator factory, and therefore should be used like so::
@config.required()
def MY_KEY(val: str) -> str:
return val
Key can be overridden manually with the ``key`` parameter.
Docstring can be overridden manually with the ``doc`` parameter.
"""
def _decorator(configurable: ct.ResolverRequired) -> ct.TYPE:
nonlocal key
nonlocal doc
if not key:
log.debug("Determining key...")
key = self._find_resolver_key(configurable)
log.debug(f"Key is: {key!r}")
log.debug("Creating required item...")
item: ct.TYPE = self._create_proxy_required(key, configurable)
log.debug("Item created successfully!")
log.debug("Registering item in the configuration...")
self.register(key, item, doc if doc is not None else configurable.__doc__)
log.debug("Registered successfully!")
# Return the created item, so it will take the place of the decorated function
return item
return _decorator
def optional(self, key: t.Optional[str] = None, doc: t.Optional[str] = None) -> t.Callable[[ct.ResolverOptional], ct.TYPE]:
def optional(self, key: t.Optional[str] = None, doc: t.Optional[str] = None) -> ct.ProxyOptional:
"""
Mark a function as a resolver for a required configuration value.
@ -176,8 +139,45 @@ class Configuration:
return _decorator
def required(self, key: t.Optional[str] = None, doc: t.Optional[str] = None) -> ct.ProxyRequired:
"""
Mark a function as a resolver for a required configuration value.
It is a decorator factory, and therefore should be used like so::
@config.required()
def MY_KEY(val: str) -> str:
return val
Key can be overridden manually with the ``key`` parameter.
Docstring can be overridden manually with the ``doc`` parameter.
"""
def _decorator(configurable: ct.ResolverRequired) -> ct.TYPE:
nonlocal key
nonlocal doc
if not key:
log.debug("Determining key...")
key = self._find_resolver_key(configurable)
log.debug(f"Key is: {key!r}")
log.debug("Creating required item...")
item: ct.TYPE = self._create_proxy_required(key, configurable)
log.debug("Item created successfully!")
log.debug("Registering item in the configuration...")
self.register(key, item, doc if doc is not None else configurable.__doc__)
log.debug("Registered successfully!")
# Return the created item, so it will take the place of the decorated function
return item
return _decorator
# noinspection PyMethodMayBeStatic
def _find_resolver_key(self, resolver: ct.Resolver) -> str:
def _find_resolver_key(self, resolver: ct.ResolverAny) -> str:
"""
Find the key of a resolver by accessing its ``__name__``.
@ -204,7 +204,7 @@ class Configuration:
log.debug(f"No values found for {key!r}, returning None.")
return None
def _create_proxy_optional(self, key: str, resolver: ct.ResolverOptional) -> lazy_object_proxy.Proxy:
def _create_proxy_optional(self, key: str, resolver: ct.ResolverOptional) -> ct.TYPE:
"""
Create, from a resolver, a proxy tolerating non-specified values.
"""
@ -234,7 +234,7 @@ class Configuration:
else:
raise errors.MissingValueError(key)
def _create_proxy_required(self, key: str, resolver: ct.ResolverRequired) -> lazy_object_proxy.Proxy:
def _create_proxy_required(self, key: str, resolver: ct.ResolverRequired) -> ct.TYPE:
"""
Create, from a resolver, a proxy intolerant about non-specified values.
"""
@ -284,7 +284,7 @@ class Configuration:
@click.command()
def root():
click.secho(f"=== Configuration ===", fg="bright_white", bold=True)
click.secho(f"===== Configuration =====", fg="bright_white", bold=True)
click.secho()
key_padding = max(map(lambda k: len(k), self.proxies.keys()))
@ -321,6 +321,8 @@ class Configuration:
click.secho(f"{doc}", fg="white")
click.secho()
click.secho(f"===== End =====", fg="bright_white", bold=True)
return root
def cli(self):

33
cfig/customtyping.py
View file

@ -1,29 +1,28 @@
"""
This module extends :mod:`typing` with the types used by :mod:`cfig`.
"""
import typing as t
TYPE = t.TypeVar("TYPE")
class Resolver(t.Protocol):
__name__: str
__doc__: str
def __call__(self, val: t.Any) -> TYPE:
...
class ResolverRequired(Resolver):
def __call__(self, val: str) -> TYPE:
...
class ResolverOptional(Resolver):
def __call__(self, val: t.Optional[str]) -> TYPE:
...
ResolverAny = t.Callable[[t.Any], TYPE]
ResolverRequired = t.Callable[[str], TYPE]
ResolverOptional = t.Callable[[t.Optional[str]], TYPE]
ProxyAny = t.Callable[[t.Callable[[t.Any], TYPE]], TYPE]
ProxyRequired = t.Callable[[t.Callable[[str], TYPE]], TYPE]
ProxyOptional = t.Callable[[t.Callable[[t.Optional[str]], TYPE]], TYPE]
__all__ = (
"TYPE",
"Resolver",
"ResolverAny",
"ResolverRequired",
"ResolverOptional",
"ProxyAny",
"ProxyRequired",
"ProxyOptional",
)

5
cfig/errors.py
View file

@ -1,3 +1,8 @@
"""
This module contains all possible exceptions occurring related to :mod:`cfig`.
"""
class CfigError(Exception):
"""
Base class for all :mod:`cfig` errors.

3
cfig/sample/__init__.py Normal file
View file

@ -0,0 +1,3 @@
"""
This module contains an example usage of :mod:`cfig`.
"""

63
cfig/sample/__main__.py
View file

@ -1,63 +0,0 @@
"""
Sample configuration module using :mod:`cfig`.
"""
import cfig
import typing
config = cfig.Configuration()
@config.required()
def MY_FAVOURITE_STRING(val: str) -> str:
"""
Your favourite string!
"""
return val
@config.optional()
def MY_OPTIONAL_STRING(val: typing.Optional[str]) -> str:
"""
Your favourite string, but optional!
"""
return val or ""
@config.required()
def MY_REQUIRED_INT(val: str) -> int:
"""
Your favourite integer!
"""
try:
return int(val)
except ValueError:
raise cfig.InvalidValueError("Not an int.")
@config.required()
def MY_FAVOURITE_EVEN_INT(val: str) -> int:
"""
Your favourite even number!
"""
try:
n = int(val)
except ValueError:
raise cfig.InvalidValueError("Not an int.")
if n % 2:
raise cfig.InvalidValueError("Not an even int.")
return n
@config.required(key="KEY_NAME")
def VAR_NAME(val: str) -> str:
"""
This config value looks for a key in the configuration sources but is available at a different key to the programmer.
"""
return val
if __name__ == "__main__":
config.cli()

116
cfig/sample/definition.py Normal file
View file

@ -0,0 +1,116 @@
"""
This module contains the definition of an example :class:`~cfig.config.Configuration` using :mod:`cfig`.
"""
# Import the cfig module, so it may be used in the file
import cfig
# Alias the typing module as t, so it can be used faster
import typing as t
# Create a new configuration using the default sources:
# - environment variables: EXAMPLE_VALUE
# - contents of the files specified in environment variables whose keys are suffixed by _FILE: EXAMPLE_VALUE_FILE
config = cfig.Configuration()
# Create a new proxy for a required configuration value
# It will behave in almost the same way as the object it is proxying
# The main differences are that:
# - "is" comparisions won't work: <Proxy None> is not None
# - some additional magic methods will be available, but you don't need to bother with them
@config.required()
def EXAMPLE_STRING(val: str) -> str:
# The proxy is a function whose name will be used as key to access its value
# Since this function is named EXAMPLE_STRING, and we are using the default sources, it will try to access in order:
# - the EXAMPLE_STRING environment variable
# - the file at the path specified in the EXAMPLE_STRING_FILE environment variable
# The docstring of the function will be used to inform the user of what should be in this configuration option.
"""
An example string: since this is an example, you can enter anything here, as it won't be used!
It will have to be *something*, though.
"""
# The code of the function will be used to process the "raw" value obtained from the sources
# Since we do not want to alter the obtained value any further, we'll just return it
return val
# Since the EXAMPLE_STRING proxy was defined as required, it will raise an error if no value is found at any source.
# Optional proxies exist, though!
@config.optional()
def EXAMPLE_NUMBER(val: t.Optional[str]) -> int:
"""
An example number: again, since this is an example, it will not matter what value you will set it to.
If you do not set it to anything, it will default to 0.
"""
# Let's default to 0 in case the user doesn't pass any value
if val is None:
return 0
# Otherwise, let's try to parse the value as an int
try:
return int(val)
# It's possible that the user entered an invalid number, though, so let's handle that case
except ValueError:
# User errors are be handled explicitly so that the user knows it's not the programmer's fault
raise cfig.InvalidValueError("Not an int.")
# And that's it!
# Let's make some more proxies as examples with no comments inbetween
# So you can have an easier idea of how cfig configs are made
@config.required()
def TELEGRAM_BOT_TOKEN(val: t.Optional[str]) -> str:
"""
The token of the Telegram bot to login as.
Obtain one at https://t.me/BotFather !
"""
try:
_id, _proper_token = val.split(":", 1)
except ValueError:
raise cfig.InvalidValueError("Not a Telegram bot token.")
return val
@config.required()
def DISCORD_CLIENT_SECRET(val: t.Optional[str]) -> str:
"""
The OAuth2 client secret of the Discord application to use.
Obtain one at https://discord.com/developers/applications !
"""
return val
# Proxies are lazily evaluated and executed only once, so you may use them to perform slower synchronous tasks
# For example, connecting to a database!
# They may be used even in global contexts such as in Flask or FastAPI apps without compromising their importability!
# Let's make a fake create_engine function so that I don't have to add sqlalchemy to cfig's dependencies
def create_engine(uri):
...
# Since the user inputs an URI, while we are creating a database
# We might want to use separate names for the "user-side" and the "programmer-side"
# We can specify the "user-side" name in the decorator
# And we can additionally specify the docstring
@config.required(key="DATABASE_URI", doc="The URI of the database to use.")
def DATABASE_ENGINE(val: str):
return create_engine(uri=val)
# Finally, let's configure the config CLI
# Let's run the CLI only if this specific script is explicitly run
if __name__ == "__main__":
# Then, pass the control to click
# Argv will be automatically read and handled
config.cli()
# Please note that this will raise an error if the "cli" extra is not installed!

15
cfig/sample/usage.py Normal file
View file

@ -0,0 +1,15 @@
"""
This module contains an example of how to use the values defined in a cfig definition module.
"""
from .definition import EXAMPLE_STRING, EXAMPLE_NUMBER, DATABASE_ENGINE
if __name__ == "__main__":
print(f"Hey! The example string you entered was {EXAMPLE_STRING}!")
# IDEA seems to be a bit confused here
# noinspection PyTypeChecker
print(f"And the square of the example number was {EXAMPLE_NUMBER ** 2}!")
print(f"We should do something with that {DATABASE_ENGINE} we created...")

4
cfig/sources/base.py
View file

@ -1,3 +1,7 @@
"""
This module defines the :class:`.Source` abstract class.
"""
import abc
import typing as t

4
cfig/sources/env.py
View file

@ -1,3 +1,7 @@
"""
This module defines the :class:`.EnvironmentSource` :class:`~cfig.sources.base.Source`.
"""
import os
import typing as t
from cfig.sources.base import Source

4
cfig/sources/envfile.py
View file

@ -1,3 +1,7 @@
"""
This module defines the :class:`.EnvironmentFileSource` :class:`~cfig.sources.base.Source`.
"""
import typing as t
from cfig.sources.env import EnvironmentSource

85
docs/index.rst
View file

@ -2,58 +2,57 @@
cfig
####
.. automodule:: cfig
The :mod:`cfig` package provides a simple but powerful configuration manager for Python applications.
Classes
Goals
=====
A goal is to allow easy integration of an application with multiple configuration standards, such as environment
variables, dotenv files, and Docker Secrets files.
.. code-block:: python
@config.required()
def SECRET_KEY(val: str) -> str:
"Secret string used to manage tokens."
return val
Another goal is to provide informative error messages to the user who is configuring the application, so that they may
understand what they are doing wrong and fix it immediately.
.. code-block:: console
$ python -m cfig.sample
=== Configuration ===
SECRET_KEY → Required, but not set.
Secret string used to manage HTTP session tokens.
HIDDEN_STRING = 'amogus'
A string which may be provided to silently print a string to the console.
Finally, the last goal is having useful typing for developers using :mod:`cfig`, allowing them to make full use of the
features of their IDEs.
.. image:: example-typing.png
Example
=======
:mod:`cfig.config`
------------------
.. automodule:: cfig.config
If you'd like to learn how to use :mod:`cfig` hands-on,
read `the source code of the cfig.sample module <https://github.com/Steffo99/cfig/tree/main/cfig/sample>`_!
:mod:`cfig.errors`
------------------
.. automodule:: cfig.errors
:show-inheritance:
Built-in sources
----------------
:mod:`cfig.sources.base`
~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: cfig.sources.base
:mod:`cfig.sources.env`
~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: cfig.sources.env
:show-inheritance:
:mod:`cfig.sources.envfile`
~~~~~~~~~~~~~~~~~~~~~~~~~~~
.. automodule:: cfig.sources.envfile
:show-inheritance:
Table of contents
=================
.. error::
I think I broke Sphinx, since the table of contents doesn't seem to show up...
Pages of this documentation
===========================
.. toctree::
terminology
reference
Other tables and links
======================

38
docs/reference.rst Normal file
View file

@ -0,0 +1,38 @@
#############
API reference
#############
.. automodule:: cfig
:mod:`cfig.config`
------------------
.. automodule:: cfig.config
:mod:`cfig.errors`
------------------
.. automodule:: cfig.errors
:show-inheritance:
:mod:`cfig.sources.base`
------------------------
.. automodule:: cfig.sources.base
:mod:`cfig.sources.env`
-----------------------
.. automodule:: cfig.sources.env
:show-inheritance:
:mod:`cfig.sources.envfile`
---------------------------
.. automodule:: cfig.sources.envfile
:show-inheritance:

29
docs/terminology.rst Normal file
View file

@ -0,0 +1,29 @@
###########
Terminology
###########
In this documentation, the following terms are used:
.. glossary::
Key
The name of a configuration value, usually in SCREAMING_SNAKE_CASE.
For example, ``PATH``, the name of the environment variable.
Value
A single non-processed configuration value in :class:`str` form.
For example, the raw string value of an environment variable.
Source
A possible origin of configuration values, such as the environment, or a file.
Proxy
An object used to lazily and transparently resolve and cache values.
After resolving a value, it behaves in almost completely the same way as the object it cached.
Resolver
A function taking in input a value originating from a source, and emitting in output its processed representation.
For example, a resolver may be the :class:`int` class, which converts the value into an integer.
Configuration
A collection of proxies.

2
pyproject.toml
View file

@ -1,6 +1,6 @@
[tool.poetry]
name = "cfig"
version = "0.1.0"
version = "0.2.0"
description = "A configuration manager for Python"
authors = ["Stefano Pigozzi <me@steffo.eu>"]
license = "MIT"