4.4 mConfig Consolidated Design Document¶
4.4.1 Overview¶
mConfig is a modular configuration library for Java, designed for flexibility, extensibility, and standardized configuration management across different operating systems and environments.
4.4.1.1 Multi-Platform and Environment Portability Goal¶
While the primary implementation is in Java, mConfig is designed
with future ports to other languages (e.g., C, Rust, Python) and varied deployment environments in mind.
- Abstraction over Platform Specifics: Core logic (in mConfigCore) MUST
remain agnostic of specific packaging formats (like JARs) or environment-specific APIs (like Java ClassLoaders).
- Embedded and Filesystem-less Systems: mConfigCore is designed to be fully functional on embedded systems where a traditional filesystem might be absent.
- No Filesystem Assumption: Core logic MUST NOT assume the presence of a filesystem or writeable persistent storage.
- Modular IO: All filesystem-specific operations (like reading/writing configuration files or exporting schemas to local storage) are isolated in separate modules (like mConfigSourceFilesystem).
- Plugin-based Initialization: Administrative tasks requiring filesystem access (e.g., auto-exporting schemas) use the ConfigFactoryComponent SPI, ensuring they only run if the relevant modules are present.
- Module Isolation: Platform-specific logic (e.g., scanning Java classpath for resources) must be isolated in dedicated modules (like mConfigSourceJAR).
- Standardized Interfaces: Use generic interfaces for discovery and storage
to ensure that the core behavior can be mirrored in other language environments
without significant architectural changes.
Core Architectural Principle: Layered Stack¶
Configurations are resolved through a stack of layers, prioritized by scope specificity. Cache (optional) -> Found Sources (ordered by scope) -> Defaults
4.4.2 Scopes and Priority¶
Scopes define the visibility and precedence of configuration data.
| Scope | Description | Priority |
|---|---|---|
| POLICY | Enforced policy settings (e.g., GPO), overriding everything else. | 1 (Highest) |
| RUNTIME | Volatile, in-memory settings for the current process. | 2 |
| SESSION | Specific to the current user session or working directory. | 3 |
| USER | Personal settings for the current user (e.g., ~/.config/). |
4 |
| APPLICATION | Settings specific to the application installation/portable root. | 5 |
| HOST | System-wide settings (e.g., /etc/opt/<company>/<app>/, %ProgramData%\<company>\<app>, HKLM). |
6 |
| CLUSTER | Settings for a cluster of hosts. | 7 |
| CLOUD | Cloud-based configurations, shared across multiple clusters. | 8 |
| ORGANIZATION | Settings shared across an organization/licensee. | 9 |
| PRODUCT | Hardcoded defaults provided by the application or modules. | 10 (Lowest) |
Tie-breaking within the same scope: Later-added layers take precedence over earlier-added ones.
4.4.2.1 Storage Type Priority¶
Resolution priority between different storage types (within the same scope) is deterministic.
Default Order (Highest to Lowest):
1. RAM (In-memory overrides)
2. secrets (Encrypted/sensitive data)
3. files (Local filesystem)
4. registry (Windows Registry - JNR-FFI)
5. registryjni (Windows Registry - JNI)
6. zookeeper (ZooKeeper - Experimental, Self-configuring)
7. JAR (Classpath resources/defaults)
This order can be customized via STORAGE_TYPE_PRIORITIES (List of IDs) and STORAGE_TYPE_ALLOW_ALL_STORAGES (Boolean).
4.4.2.2 Self-Configuration¶
mConfig supports "self-configuration" to allow the library to configure its own behavior using the same discovery mechanisms it provides to applications.
4.4.2.2.1 Library Self-Configuration (ConfigFactorySettings)¶
During ConfigFactoryBuilder.build(), mConfig core (SelfConfigurator) reads the component file
.config/<company>/<application>/mconfig.properties (without a company: .config/<application>/mconfig.properties)
for the company and application the factory is built for.
- Per component: an application often contains several components that use mConfig, each with its own factory. The file is bound to company and application, so each component configures only itself and does not change the behaviour of the others.
- Precedence: code (
setFeature()) > runtime settings (if delegated) > component file > deprecated global file. A file only fills in features that are not set yet. - Setting levels: features marked
CODE_ONLY(e.g.ALLOW_MCONFIG_RUNTIME_SETTINGS,ALLOW_APPLICATION_POLICY_LOCATIONS) are refused;COMPONENTfeatures (TEST_MODE,TESTMODE_DIRECTORIES,PERMIT_TEST_MODE) are allowed. File-set test mode is logged as a WARNING with the file's location. - Timing: before the test-mode gates and before storage discovery, so
forbidTestMode()andPERMIT_TEST_MODEcover file-set test mode, and storage settings take effect. - Format: Standard Java
.propertiesformat only. Keys (ignoring case) are matched against registered feature names. - Disabling: enabled by default; disable with
ConfigFeature.ENABLE_SELF_CONFIGURATION = falsein code. - Deprecated global file:
.config/metabit/mConfig/mconfig.propertiesapplies to every factory in the process and cannot be attributed to a component. Still read in 0.8.x with a WARNING, ordinary (ANY) features only; removed before 0.9.0.
See Config Features.
4.4.2.2.2 Networked Source Self-Configuration¶
Networked sources (like ZooKeeper) can also retrieve their own connection settings (connect string, root path, etc.) from previously loaded layers (e.g., JAR defaults or local filesystem overrides).
Resolution Order during Startup:
1. Library Self-Configuration: runtime settings, then the component's mconfig.properties (see 4.4.2.2.1) populate ConfigFactorySettings; then the test-mode gates are evaluated.
2. Core Discovery: Modules (Storages, Formats) and ConfigFactoryComponents are identified.
3. Local Source Initialization: JARConfigSource and FileConfigStorage are initialized based on STORAGE_TYPE_PRIORITIES.
4. Networked Source Initialization:
- ZooKeeperConfigStorage.init() is called.
- It requests its own configuration (default name: "zookeeper") from the factory via ctx.getFactory().getConfig("zookeeper").
- If settings are found in JAR defaults or local files, it connects and registers its locations (e.g., CLUSTER, ORGANIZATION).
- If settings are missing, it remains inactive (zero-config support).
5. 2-Phase Post-Initialization:
- postInit(ConfigFactory) is called for all ConfigStorageInterface, ConfigSecretsProviderInterface, ConfigFactoryComponent, and ConfigLoggingInterface instances.
- Execution Order:
1. ConfigStorageInterface: Storages get the first chance to configure themselves.
2. ConfigSecretsProviderInterface: Secrets providers configure themselves using the now-ready storages.
3. ConfigFactoryComponent: General administrative components (e.g., SecretsFactoryComponent) run next.
4. ConfigLoggingInterface: The final step, allowing the logger to refine its configuration using all other components.
- This allows modules to perform complex self-configuration (e.g., obtaining credentials from an already initialized secret provider) or administrative tasks (e.g., auto-exporting schemas, refining remote logging endpoints) using the fully functional ConfigFactory.
6. Finalization: The factory is ready, including any successfully activated networked layers.
See ZooKeeper Example for detailed configuration snippets.
4.4.3 Core Components¶
ConfigFactory¶
The main entry point. It manages the ConfigSearchList and instantiates Configuration objects.
ConfigSearchList¶
A prioritized list of ConfigLocation entries defining where the library should look for configurations.
ConfigLocation & ConfigSource¶
- ConfigLocation: Defines where a configuration might be found (Storage + Handle + Scope).
- ConfigSource: An instantiated
ConfigLocationthat is associated with a specificConfigLayer. It confirms the configuration exists and is readable.
ConfigStorage¶
Provides low-level access to data (e.g., FileConfigStorage, WindowsRegistryConfigSource, EnvVarConfigStorage).
ConfigFormat¶
Parses and writes data in specific formats (e.g., Properties, JSON, YAML, TOML, JSON5, INI).
For storages that hold configurations as blobs (files, JAR resources, ZooKeeper nodes), the format
reads the blob and creates the ConfigLayer.
A format is a helper of the storage that found the content: the format knows the syntax, the storage
the context (configuration, schema, whether the location is a secret store). Since 0.8.17 a format
returns a layer or throws ConfigParseException; it does not log parse problems itself. The storage
catches the exception and hands it to core's ParseProblemReporter, which decides whether the
parser's own text may be shown (see Logging: parse errors).
ConfigLayer¶
The concrete data read from one ConfigLocation, mapped to hierarchical keys (e.g., database/host) and
ConfigEntry objects. For blob-like storages (files, ZooKeeper BLOBs, JAR resources) a format produces it;
storages with structured data of their own (Windows registry, environment variables, Vault) produce it themselves.
ConfigEntry¶
A leaf node in the configuration tree, containing the value and metadata (like the source location).
4.4.4 Operational Modes¶
TEST_MODE¶
- Bypasses standard OS locations.
- Uses only explicitly provided test paths (via
TESTMODE_DIRECTORIESfeature) and standard test resources. TESTMODE_DIRECTORIESentries support theSCOPENAME ":" PATHformat.- Prevents tests from interfering with or relying on actual user configurations.
- Security: Employs a two-tier gate system (Global Static Gate + Instance Dynamic Gate).
PERMIT_TEST_MODEdefaults totrue("Productive by Default") but can be locked down per instance.
ADDITIONAL_DIRECTORIES¶
ADDITIONAL_RUNTIME_DIRECTORIESandADDITIONAL_USER_DIRECTORIESallow adding custom search paths at runtime.- Support the
SCOPENAME ":" PATHformat, allowing paths to be assigned to another scope than their default scope. Exception: POLICY. In productive mode aPOLICY:entry fails with aConfigExceptionunless the application setALLOW_APPLICATION_POLICY_LOCATIONSin code. The same guard applies to a secrets provider with scope POLICY and to POLICY sources added withConfigFactoryBuilder.addSource(). Storages' own, OS-derived POLICY locations are not affected. See Config Features. - Prepended to the search list of their respective scope, ensuring they take precedence over default locations.
SERVICE-mode (Auto-refresh & Monitoring)¶
- If enabled, every access re-checks the search list for new or updated configuration files.
- Recommended to be used with a Cache Layer to mitigate performance overhead.
- Cache entries should have a configurable TTL (Time-To-Live).
- Monitoring Support:
Configurationsupports thesubscribeToUpdates(Consumer<ConfigLocation>)mechanism. This is used by themonitorCLI command to provide real-time reporting of changes.
4.4.4.2 Self-Configuration Security¶
Self-configuration (see section 2.2.1) automatically applies settings from the classpath. In environments where the classpath might contain untrusted JARs, this feature can be disabled:
ConfigFactoryBuilder.create("metabit", "APP")
.setFeature(ConfigFeature.ENABLE_SELF_CONFIGURATION, false)
.build();
Disabling self-configuration ensures that the ConfigFactory starts with only the settings explicitly provided in code, preventing "configuration injection" via the classpath.
Independently of this switch:
- a component file applies only to its own company and application, and code always wins over it;
- no file can set CODE_ONLY features (e.g. ALLOW_MCONFIG_RUNTIME_SETTINGS, ALLOW_APPLICATION_POLICY_LOCATIONS);
- the deprecated global file cannot set COMPONENT features (TEST_MODE, TESTMODE_DIRECTORIES, PERMIT_TEST_MODE);
- refused entries are ignored with a WARNING; file-set test mode is logged as a WARNING with the file's location.
4.4.5 Reading and Writing¶
Reading¶
- Sequential search from most specific to least specific scope.
- Value conversion (String to Integer, Boolean, etc.) is handled at the
Configurationfacade level.
Writing¶
- Direct Write: Attempt to write back to the
ConfigLocationwhere the entry was found. - Scope-based Write: Specify a scope for the write operation.
- Write Fallback (Fall-forward): If writing to the preferred location fails, attempt to write to a more specific scope (e.g., if MACHINE is read-only, write to USER).
- Write Cache: Optional layer that buffers writes until a
flush()is called.
4.4.6 Security and Secrets¶
mConfig provides specialized handling for sensitive data (API keys, passwords, certificates).
4.4.6.1 Secret Entries¶
- Sensitive data should be handled via
SecretValueandSecretConfigEntryLeaf. - Values are obfuscated in memory (
char[]orbyte[]instead ofStringwhere possible). toString()implementation of secret entries is automatically redacted ([REDACTED]).ConfigFacadeImpl.getSecret(key)is the preferred API to retrieve aSecretValue.
4.4.6.2 Schema-Driven Secrets¶
A ConfigEntry is treated as a secret if its ConfigEntrySpecification (from a ConfigSchema) has the SECRET flag set.
- ConfigEntryFactory automatically creates a SecretConfigEntryLeaf if the schema marks the key as a secret.
- Intelligent Type Selection: When creating a secret entry, the factory chooses a SecretType (e.g., PLAIN_TEXT vs SYMMETRIC_KEY) based on the requested ConfigEntryType.
4.4.6.3 External Secrets Providers¶
External authentication (e.g., towards networked sources like HashiCorp Vault)
can be integrated via specialized modules implementing ConfigSecretsProviderInterface.
- Secret providers are initialized during the configuration loading phase.
- They can receive a ConfigurationPhaseWrapper to access
partially loaded configurations for their own initialization
(e.g., to read the Vault address from a local file).
4.4.7 Format Specifics¶
Java .properties¶
- Flat key-value pairs. Trims whitespace.
- Supports multi-line values with
\.
JSON / JSON5¶
- Toplevel must be an Object or Array.
- JSON5 adds support for comments, trailing commas, and unquoted keys.
YAML¶
- Supports hierarchical structures.
- Implementation must ensure secure parsing (avoiding arbitrary command execution).
INI¶
- Section-based (
[section]). - Hierarchical keys can be mapped to sections using
/(e.g.,[first/second]).
4.4.7.7 Writing Priorities¶
When writing to a configuration (e.g., via put), mConfig applies the following priorities within the requested scope(s):
- Priority 1: Update Existing Entry. The highest-priority writeable layer in the scope that already contains the key is updated.
- Priority 2: Add to Existing Layer. If the key is new to the scope, it is added to the highest-priority writeable layer already instantiated in that scope.
- Priority 3: Create New Layer. If no writeable layers exist for the scope, mConfig attempts to create a new one using the prioritized
ConfigStorageandConfigFormatsettings.
Writes are strictly scope-isolated unless WRITE_FALLBACK_ACROSS_SCOPES is enabled. For multi-scope writes, mConfig iterates from the most specific to the most generic scope, stopping at the first successful write.
4.4.8 BLOBs (Binary Data)¶
mConfig supports Binary Large Objects (certs, licenses, keys) in two distinct ways:
4.4.8.1 Hierarchical BLOBs (Keyed Leaves)¶
In hierarchical formats like JSON and YAML, binary data can exist as a leaf node with a specific key (e.g., server/certificate).
- YAML: Native support via the !!binary tag.
- JSON: Supported via Base64 encoding. If a ConfigSchema marks a key as BYTES, the JSON format will automatically decode the Base64 string into a BlobConfigEntryLeaf.
- These entries are part of the regular configuration tree and can be accessed via getValueAsBytes().
4.4.8.2 Flat BLOBs (Opaque Files)¶
Entire files can be treated as a single binary blob using the FileRawBinaryFormat (extension .bin).
- These are opaque and have no internal structure.
- The entry is always accessed using the empty string key ("").
- Requesting any other key from a flat binary layer will return null.
4.4.9 Crypto Library Interaction¶
mConfig is designed to work seamlessly with standard Java crypto libraries (JCE) and third-party libraries like BouncyCastle.
- JCE Integration:
SecretValuecan store raw bytes ofPrivateKey,PublicKey, orSecretKey. Conversion typically involveskey.getEncoded(). - BouncyCastle: Can be used to process
SecretValuecontents (e.g., parsing a PEM-encoded certificate stored as aBYTESentry). - Test Strategy: Integration tests (
SecretsJCETest,SecretsBCTest) verify that low-level binary secrets are preserved exactly as they move through the configuration stack.
4.4.10 Configuration Schemas¶
Configuration Schemas define the contract for a set of configuration entries, specifying keys, types, defaults, and validation rules.
See the detailed Configuration Schemas for information on:
- Schema structure and properties
- Future-proofing with the MANDATORY block
- JSON format variants
- Discovery and registration via ConfigSchemaRepository
4.4.10.1 Discovery and Precedence¶
Schemas are automatically discovered from the classpath (specifically within .config/ directories) if the JARConfigSource is active.
- Unified Discovery: Discovery is performed across all classpath resources, including both
mainandtestresource directories. - Precedence: If multiple schema files for the same configuration name are discovered, a "Last-One-Wins" strategy is applied based on the order returned by the
ClassLoader. In typical development environments (Maven/Gradle), this means production resources might overwrite test resources if they appear later in the classpath enumeration. - Scope Agnosticism: Unlike configuration data (layers), schemas are not scope-isolated in
TEST_MODE. Production schemas are always loaded to ensure a consistent contract.
4.4.10.2 Discovery and Binding Flow (Mermaid)¶
The following diagram illustrates how schemas are discovered by storages, registered in the factory, and finally bound to configuration instances.
Layered Configuration (Mermaid)¶
4.4.11 Extending mConfig¶
4.4.11.1 Adding New Storages¶
This is intended for library maintainers extending mConfig with new backends (e.g., databases, cloud stores).
When implementing a new ConfigStorage:
- Module Creation: Create
mConfigSource<Name>module withConfigStorageInterfaceimpl. - init(ctx): Populate
ctx.getSearchList()withConfigLocationImplfor scopes/formats/paths. - Read Layers: In
updateConfigurationLayers, load/parse entries intoLayeredConfiguration. - Classpath Order: Auto-discovered; control via
ConfigFeature.STORAGE_TYPE_PRIORITIES. - Testing: Add unit/integration tests; verify with full
mvn clean test.
Examples: FileConfigStorage, JARConfigSource, WindowsRegistryConfigSource.
In case a specific config format is to be supported as well, it can go into the same module.
4.4.11.2 Adding New ConfigFeatures¶
This is not for library consumers; only project owners should need to perform this step.
When adding a new constant to the ConfigFeature enum in mConfigCore, or to a module-specific features class, you MUST perform the following steps to ensure technical consistency and avoid runtime errors:
- Constant Definition: Add the new constant to the
ConfigFeatureenum or create a newConfigFeatureInterfaceinstance. - Type Initialization: You MUST assign a
ValueTypeto thevalueTypefield (or return it viagetValueType()).- Example:
MY_FEATURE.valueType = ValueType.STRING;
- Example:
- Default Values (Optional): If the feature should have a default value, assign it to the
defaultValuefield in the samestaticblock.- Example:
MY_FEATURE.defaultValue = Boolean.TRUE;
- Example:
- Verification: Run the
ConfigFeatureTypesTestunit test to ensure that all features have a valid type assigned. A missing type will cause aNullPointerExceptionwhen the feature is accessed viaConfigFactorySettings.
4.4.12 Best Practices¶
Classpath Layout¶
- Production Defaults:
src/main/resources/.config/<company>/<app>/<config>.<ext>(JARConfigSource). - Test Mode:
src/test/resources/.config/<company>/<app>/+config/{SCOPE}(overrides project). - Schemas: Same path +
.mconfig-schema.json.
Logging¶
- Implement
ConfigLoggingInterfacefor structured logs (SLF4J/JUL/Logback). - Avoid:
System.out.println(blocks UI/threads). - Secrets auto-redacted.
Secrets Handling¶
- Retrieve:
SecretValue sv = config.getSecret(key)(schema-marked SECRET). - Use:
byte[] data = sv.getValue();sv.erase()post-use (secure wipe). - Storage: Formats preserve
SecretValue; logs masked.
See Handling Secrets.