Configuration File Architecture

pyrig's ConfigFile system provides automatic creation, validation, and updating of project configuration files through a declarative class-based architecture.

Core Concepts

ConfigFile Base Class

The ConfigFile abstract base class is the foundation of the configuration system. All config files inherit from it or one of its format-specific subclasses.

When you run pyrig mkroot or pyrig init, all ConfigFile subclasses are automatically discovered and initialized, creating the complete project configuration.

Note: BuilderConfigFile is a special subclass for artifact generation (e.g., creating executables). It repurposes the ConfigFile interface for build operations rather than configuration file management. Builders are discovered separately from pyrig.rig.builders and invoked via pyrig build. See Builder Documentation for details.

Required Methods

Every ConfigFile subclass must implement:

Note: ConfigT is the type parameter - ConfigDict for DictConfigFile, ConfigList for ListConfigFile, or ConfigData for base classes like YamlConfigFile and JsonConfigFile. Format-specific subclasses provide implementations for _load(), _dump(), and extension(). ConfigDict and ConfigList are type aliases for nested dicts and lists with string keys (just dict[str, Any] and list[Any]), so you can simply use dicts and lists to define your configuration structure in _configs() or when overriding other methods.

Subclasses implement _load() and _dump() (internal methods). Users call load() and dump() (public API with caching).

Caching System

Both load() and configs() use @functools.cache to avoid redundant operations:

• First call: Executes _load() or _configs() and caches the result
• Subsequent calls: Returns cached data without re-execution
• Cache invalidation: dump() clears the load() cache before and after writing (the configs() cache is never cleared since config structure shouldn't change at runtime)

This provides significant performance improvements when config files are accessed multiple times, while ensuring data consistency after writes.

Caching Pitfall: Never Mutate Cached Results

Due to caching, mutating the object returned by load() or configs() creates subtle bugs:

# ❌ WRONG - mutates cached object
loaded = MyConfigFile.I.load()
loaded["key"] = "new_value"  # This corrupts the cache!

# ❌ WRONG - in-place list modification
deps = MyConfigFile.I.load()["dependencies"]
deps.append("new-dep")  # Mutates cached list

# ✓ CORRECT - create a new structure
loaded = MyConfigFile.I.load()
new_config = {**loaded, "key": "new_value"}

# ✓ CORRECT - copy before modifying
import copy
loaded = copy.deepcopy(MyConfigFile.I.load())
loaded["key"] = "new_value"

# ✓ CORRECT - create new list
deps = [*MyConfigFile.I.load()["dependencies"], "new-dep"]

This is especially important for ConfigFiles that call load() inside _configs(). For example, PyprojectConfigFile.I._configs() reads the existing pyproject.toml (via calling load()) to preserve user-defined values. If _configs() mutates the loaded data, the cache becomes corrupted and subsequent calls return incorrect values.

Rule: Always treat load() and configs() results as read-only. Return new structures instead of modifying in place.

Validation Process

When a ConfigFile is instantiated, it follows this process:

Note: Config files are grouped by priority and initialized by group (highest priority first). Within each priority group, files are initialized in parallel using ThreadPoolExecutor for improved performance.

Validation Logic

A config file is considered correct if:

1. Empty file - User opted out of this configuration file
2. Superset validation - Actual config contains all expected keys/values

The validation recursively checks that the expected configuration is a subset of the actual configuration. Only use lists and dicts to define your configuration structure.

Smart Merging

When configs are missing or incorrect, the system intelligently merges them:

• Dict values - Missing keys are added. Important: Keys with incorrect values are overwritten with the expected values from _configs(). This ensures required configuration is always correct.
• List values - Missing items are inserted at the correct index
• User additions - Preserved during merge (extra keys in dicts, extra items in lists)

Customizing default values: To change default values, subclass the specific config file and override the _configs() method. Keep the class name the same (filename is derived from it) unless you also override filename(). Import the base class using: from pyrig.rig.configs.some.config_file import MainConfigFile as BaseMainCF and subclass it.

Automatic Discovery

ConfigFile subclasses are automatically discovered across all packages depending on pyrig:

Discovery uses discover_subclasses_across_dependents with discard_parents=True, meaning only leaf implementations are initialized. This way you can subclass a config file and only your subclass will be initialized. This makes it easy to adjust any config file to your liking.

Note: I highly recommend doing any changes to config files via a subclass and not directly in the file itself. This way you can regenerate the file at any time and it will not overwrite your changes. Also this way packages that will later depend on your package will automatically get your config file adjustments as well.

Validation Order

Config files are initialized during pyrig mkroot or pyrig init using a priority-based system that balances correctness with performance.

Priority System

Each ConfigFile subclass can override priority() to specify Validation order:

def priority(self) -> float:
    """Return priority for this config file.

    Higher numbers are processed first. 
    Return 0 (default) if order doesn't matter.
    """
    return 0  # Default: no priority

Priority values in pyrig:

• LicenseConfigFile: 30 (must exist before pyproject.toml for license detection)
• PyprojectConfigFile: 20 (must exist before other configs that read from it)
• ConfigsInitConfigFile: 10 (creates package structure)
• FixturesInitConfigFile: 10 (must exist before conftest.py)
• ToolsInitConfigFile: 10 (creates tools package structure)
• All others: 0 (no specific order required)

How Validation Works

The validation process groups config files by priority and processes each group:

1. Grouping: All config files are grouped by their priority value
2. Sequential group processing: Priority groups are processed in order (highest priority first)
3. Parallel within groups: All files within the same priority group are initialized in parallel using ThreadPoolExecutor

Example execution order:

• Priority 30 group (LICENSE) - Initialized first
• Priority 20 group (pyproject.toml) - Initialized second
• Priority 10 group (ConfigsInitConfigFile, FixturesInitConfigFile) - Initialized third, both in parallel
• Priority 0 group (all other configs) - Initialized last, all in parallel

Key methods:

• ConfigFile.validate_all_subclasses() - validate all discovered config files
• ConfigFile.subclasses() - Discover all config files (sorted by priority)

Performance Benefits

The hybrid priority-based approach provides:

• Correctness - Dependencies are respected through priority ordering
• Performance - Files without dependencies validate concurrently
• Flexibility - Same priority = can run in parallel, different priority = guaranteed order

This is significantly faster for projects with many config files while ensuring dependencies (like LICENSE before pyproject.toml) are always met.

Format-Specific Subclasses

These subclasses implement common methods for specific file formats, simplifying ConfigFile creation. They provide implementations for _load(), _dump(), and extension() so you only need to define the file location and expected content.

DictConfigFile

Intermediate base class for dict-based configuration files. Most config files use this as their base (directly or indirectly via format-specific subclasses).

from pathlib import Path
from pyrig.rig.configs.base.base import ConfigDict
from pyrig.rig.configs.base.dict_cf import DictConfigFile

class MyConfigFile(DictConfigFile):
    def parent_path(self) -> Path:
        return Path("config")

    def extension(self) -> str:
        return "conf"

    def _load(self) -> ConfigDict:
        # Custom loading logic
        return {}

    def _dump(self, config: ConfigDict) -> None:
        # Custom dumping logic
        pass

    def _configs(self) -> ConfigDict:
        return {"key": "value"}

Inherits from: ConfigFile[ConfigDict]

Use when: Creating custom dict-based formats not covered by existing subclasses like TomlConfigFile, JsonConfigFile, etc.

ListConfigFile

Intermediate base class for list-based configuration files (e.g., .gitignore).

from pathlib import Path
from pyrig.rig.configs.base.base import ConfigList
from pyrig.rig.configs.base.list_cf import ListConfigFile

class MyListConfigFile(ListConfigFile):
    def parent_path(self) -> Path:
        return Path(".")

    def extension(self) -> str:
        return "list"

    def _load(self) -> ConfigList:
        # Custom loading logic
        return []

    def _dump(self, config: ConfigList) -> None:
        # Custom dumping logic
        pass

    def _configs(self) -> ConfigList:
        return ["item1", "item2"]

Inherits from: ConfigFile[ConfigList]

Use when: Creating config files where the top-level structure is a list rather than a dict (e.g., ignore files, line-based configs).

JsonConfigFile

For JSON configuration files using Python's json module:

from pathlib import Path
from pyrig.rig.configs.base.base import ConfigDict
from pyrig.rig.configs.base.json import JsonConfigFile

class MyConfigFile(JsonConfigFile):
    def parent_path(self) -> Path:
        return Path("config")

    def _configs(self) -> ConfigDict:
        return {"key": "value"}

Creates config/my_config.json.

YamlConfigFile

For YAML configuration files using PyYAML:

from pathlib import Path
from pyrig.rig.configs.base.base import ConfigDict
from pyrig.rig.configs.base.yaml import YamlConfigFile

class MyConfigFile(YamlConfigFile):
    def parent_path(self) -> Path:
        return Path("config")

    def _configs(self) -> ConfigDict:
        return {"key": "value"}

Creates config/my_config.yaml.

TomlConfigFile

For TOML files using tomlkit (preserves formatting):

from pathlib import Path
from pyrig.rig.configs.base.base import ConfigDict
from pyrig.rig.configs.base.toml import TomlConfigFile

class MyConfigFile(TomlConfigFile):
    def parent_path(self) -> Path:
        return Path(".")

    def _configs(self) -> ConfigDict:
        return {"tool": {"myapp": {"setting": "value"}}}

Creates my_config.toml with pretty formatting.

StringConfigFile

For plain text files with required content. "String" here means files that cannot be represented as structured data (dicts/lists). This includes .py, .txt, .md files, etc. Files like .yaml, .json, .toml that can be represented as dicts/lists should use their specific subclasses instead. It adds by default an empty line at the end of the file if not already present.

from pathlib import Path
from pyrig.rig.configs.base.string_ import StringConfigFile

class MyConfigFile(StringConfigFile):
    def parent_path(self) -> Path:
        return Path(".")

    def lines(self) -> list[str]:
        return ["# Required header"]

    def extension(self) -> str:
        return "someext"

Creates my_config.someext. The file is considered correct if it contains the required content anywhere within it. You can add your own content before or after the required content.

PythonConfigFile

For Python source files:

from pathlib import Path
from pyrig.rig.configs.base.python import PythonConfigFile

class MyConfigFile(PythonConfigFile):
    def parent_path(self) -> Path:
        return Path("myapp/src")

    def lines(self) -> list[str]:
        return ['"""Module docstring."""', '', 'def main():', '    pass']

Creates myapp/src/my_config.py.

WorkflowConfigFile

For GitHub Actions workflow files in .github/workflows/:

from pyrig.rig.configs.base.base import ConfigDict
from pyrig.rig.configs.base.workflow import WorkflowConfigFile

class MyWorkflowConfigFile(WorkflowConfigFile):
    def workflow_triggers(self) -> ConfigDict:
        """Define when the workflow runs."""
        triggers = super().workflow_triggers()
        triggers.update(self.on_push())  # Trigger on push
        return triggers

    def jobs(self) -> ConfigDict:
        """Define the workflow jobs."""
        return {
            "my-job": {
                "runs-on": self.UBUNTU_LATEST,
                "steps": [
                    self.step_checkout_repository(),
                    self.step_setup_python(),
                    {"name": "Run tests", "run": "pytest"}
                ]
            }
        }

Creates .github/workflows/my_workflow.yml. The WorkflowConfigFile class provides many helper methods for building jobs, steps, triggers, and matrix strategies. See the workflow documentation for details.

Advanced Subclasses

CopyModuleConfigFile

Copies entire module content from pyrig to your project:

from types import ModuleType
from pyrig.rig.configs.base.copy_module import CopyModuleConfigFile
from pyrig import main

class MainConfigFile(CopyModuleConfigFile):
    def src_module(self) -> ModuleType:
        return main

Creates myapp/main.py with full content from pyrig.main.

CopyModuleOnlyDocstringConfigFile

Copies only the docstring, allowing custom implementation:

from types import ModuleType
from pyrig.rig.configs.base.copy_module_docstr import (
    CopyModuleOnlyDocstringConfigFile,
)
from pyrig.rig.cli import shared_subcommands

class SharedSubcommandsConfigFile(CopyModuleOnlyDocstringConfigFile):
    def src_module(self) -> ModuleType:
        return shared_subcommands

Creates myapp/rig/cli/shared_subcommands.py with only the docstring.

InitConfigFile

Creates __init__.py files with docstrings:

from types import ModuleType
from pyrig.rig.configs.base.init import InitConfigFile
from pyrig.rig import configs

class ConfigsInitConfigFile(InitConfigFile):
    def src_module(self) -> ModuleType:
        return configs

Creates myapp/rig/configs/__init__.py with the docstring from pyrig.rig.configs.

MirrorTestConfigFile

Creates test files that mirror source module structure with skeleton tests:

from types import ModuleType
from pyrig.rig.tests.mirror_test import MirrorTestConfigFile
import myapp.core

class CoreMirrorTest(MirrorTestConfigFile):
    def src_module(self) -> ModuleType:
        return myapp.core

Creates tests/test_myapp/test_core.py with test skeletons for all functions, classes, and methods in myapp.core. Existing tests are preserved; only missing skeletons are added. Used internally by the mktests command and autouse fixtures for automatic test generation.

Customizing Test Generation

Subclass MirrorTestConfigFile in your project's rig/tests/ directory to customize skeleton generation. The L class property ensures your subclass is used. For example, to change the skeleton for test functions:

# myapp/rig/tests/mirror_test.py
from pyrig.rig.tests.mirror_test import MirrorTestConfigFile as BaseMirrorTest

class MirrorTestConfigFile(BaseMirrorTest):
    def test_func_skeleton(self, test_func_name: str) -> str:
        return f'''
def {test_func_name}() -> None:
    """Test function for {test_func_name.removeprefix("test_")}."""
    pytest.skip("TODO: implement test")
'''

Filename Derivation

Filenames are automatically derived from class names:

The system:

1. Removes abstract parent class suffixes (ConfigFile, YamlConfigFile, etc.)
2. Converts to snake_case
3. Adds the file extension

Override filename() for custom names:

def filename(self) -> str:
    return ""  # Creates ".env" instead of "dot_env.env"

Opt-Out Mechanism

Users can opt out of any config file by emptying it. An empty file is considered "unwanted" and validation passes. There are few exceptions (e.g., README.md is always required). You can override is_unwanted() to customize this behavior.

Note: Deleting a file will just lead to it being recreated. Emptying it is the only way to opt out.

Custom Validation

Override is_correct() for custom validation logic, pyrig does this in many of its own config files:

def is_correct(self) -> bool:
    """Check if config contains required structure."""
    content = self.file_content()
    return super().is_correct() or (
        "required_string" in content
        and "another_required_string" in content
    )

Best Practices

1. Inherit from format-specific classes - Use YamlConfigFile, TomlConfigFile, etc.
2. Keep configs minimal - Only specify required values
3. Allow user customization - Validation checks for supersets, not exact matches
4. Use priority/ordered lists - For configs with dependencies
5. Override is_correct() carefully - Ensure it allows user additions
6. Document expected structure - In docstrings and comments

Example: Complete Custom Config

from pathlib import Path
from pyrig.rig.configs.base.base import ConfigDict
from pyrig.rig.configs.base.yaml import YamlConfigFile

class DatabaseConfigFile(YamlConfigFile):
    """Configuration for database connection settings."""

    def parent_path(self) -> Path:
        """Place in config/ directory."""
        return Path("config")

    def _configs(self) -> ConfigDict:
        """Required database configuration."""
        return {
            "database": {
                "host": "localhost",
                "port": 5432,
                "name": "myapp_db"
            }
        }

    def is_correct(self) -> bool:
        """Ensure required keys exist."""
        config = self.load()
        required_keys = {"host", "port", "name"}
        actual_keys = set(config.get("database", {}).keys())
        return super().is_correct() or required_keys.issubset(actual_keys)

Place in myapp/rig/configs/database.py and it will create config/database.yaml with the required structure.