@drillsense/fed-npt-prediction
flwr new @drillsense/fed-npt-predictionFed-NPT: Federated Non-Productive Time Prediction
By DrillSense — The Anti-Platform Platform for Drilling Intelligence
Federated NPT prediction for oil and gas operators. Each client is an operator; the server builds a global NPT classifier using FedProx and Differential Privacy — no operator ever shares well-level NPT logs or cost data. Uses demo data derived from the open-source 3W dataset (Petrobras, Apache-2.0).
Note on data loading: The 3W dataset is not available on Hugging Face, so this app uses a local CSV/Parquet file instead of flwr_datasets. A built-in CLI (npt-prepare-data) is provided to build the demo file from the 3W raw Parquet files.
Why Federated Learning for NPT?
NPT data is among the most commercially sensitive records an operator produces — it reveals equipment failure rates, crew inefficiency, geological surprises, and cost overruns per well. No operator will share this with a third party. But every operator would benefit from a model that predicts NPT before it happens.
Federated learning solves this: each operator trains on their own data locally, sharing only model weight updates. Two additional Flower capabilities make this production-ready:
- FedProx — adds a proximal regularisation term that prevents local models from drifting too far from the global model. Essential when operators have heterogeneous NPT distributions (different geology, equipment, practices).
- Differential Privacy (server-side adaptive clipping) — adds calibrated Gaussian noise with an adaptive clipping norm to the aggregated updates. The server learns nothing about any individual operator's NPT patterns. Configurable via noise-multiplier.
Fetch the App
Install Flower:
pip install flwr
Fetch the app from Flower Hub:
flwr new @drillsense/fed-npt-prediction
This will create a new directory called fed-npt-prediction with the following structure:
fed-npt-prediction/
├── fed_npt_prediction/
│ ├── __init__.py
│ ├── client_app.py # Flower ClientApp — one per operator
│ ├── server_app.py # Flower ServerApp — FedProx + DP aggregation
│ ├── task.py # Model, training, data loading, validation
│ └── prepare_data.py # Build demo data from 3W dataset (CLI or module)
├── data/ # Demo data (auto-downloaded from Zenodo on first run)
│ ├── simulation/
│ │ └── demo_data.csv # 2,228 wells for simulation
│ └── deployment/
│ ├── client_1_demo_data.csv # 1,114 wells — Operator 1
│ └── client_2_demo_data.csv # 1,114 wells — Operator 2
├── pyproject.toml
└── README.md
Demo data sets
Demo datasets are derived from the 3W dataset (Petrobras, Apache-2.0) and hosted on Zenodo:
For simulation — the app automatically downloads demo_data.csv from Zenodo the first time you run it if the file is not found locally. No manual download needed.
For deployment — download the per-operator demo files to data/deployment/:
wget "https://zenodo.org/records/18945258/files/client_1_demo_data.csv?download=1" -O data/deployment/client_1_demo_data.csv wget "https://zenodo.org/records/18945258/files/client_2_demo_data.csv?download=1" -O data/deployment/client_2_demo_data.csv
Each file is a well-level table: 20 summary features (mean + std of 10 3W sensor columns) + a binary label column (0 = normal operation, 1 = NPT event). The simulation file contains 2,228 wells; each deployment file contains 1,114 wells.
Want to regenerate from the raw 3W data? Clone petrobras/3W and run:
npt-prepare-data --3w-dir /path/to/3W/dataset --binary --out data/simulation/demo_data.csv
Run the App
Simulation
This app runs with 5 virtual SuperNodes by default, each representing a different operator. No Flower Configuration changes are required — the default local-simulation federation handles everything.
Install dependencies:
cd fed-npt-prediction && pip install -e .
Run with default settings (10 rounds, FedProx + DP enabled):
flwr run .
Override settings:
flwr run . --run-config "num-server-rounds=20 proximal-mu=0.05 noise-multiplier=0.3"
To run without DP (plain FedProx):
flwr run . --run-config "noise-multiplier=0.0"
TIP — Check the Simulation Engine documentation to learn more about running with more virtual SuperNodes or configuring CPU/GPU allocation.
Deployment
In Deployment mode each real SuperNode loads its own local data via node-config. The partition-id / num-partitions keys are absent, so the app automatically switches to local data loading.
Download the per-operator demo files (see above), then launch each SuperNode with its data path:
# SuperNode 1 — Operator 1's data flower-supernode \ --insecure \ --superlink <SUPERLINK-FLEET-API> \ --node-config="data-path=/path/to/data/deployment/client_1_demo_data.csv" # SuperNode 2 — Operator 2's data flower-supernode \ --insecure \ --superlink <SUPERLINK-FLEET-API> \ --node-config="data-path=/path/to/data/deployment/client_2_demo_data.csv"
Then launch the run:
flwr run . <SUPERLINK-CONNECTION> --stream
TIP — Follow the Deployment Engine guide for full setup, including TLS-enabled communications and SuperNode authentication.
Configuration reference
| Key | Default | Description |
|---|---|---|
| num-server-rounds | 10 | Federated rounds |
| batch-size | 32 | Local batch size |
| local-epochs | 2 | Epochs per client per round |
| learning-rate | 0.001 | Adam learning rate |
| test-fraction | 0.2 | Fraction of local data held out for validation |
| sim-data-path | data/simulation/demo_data.csv | CSV/Parquet path for simulation (auto-downloaded if missing) |
| num-partitions | 5 | Virtual clients in simulation |
| model-save-path | output/final_npt_model.pt | Where the server saves the final global model |
| weight-decay | 1e-5 | L2 regularisation for Adam |
| seed | 42 | Random seed for reproducibility |
| early-stopping-patience | 3 | Stop local training if val loss does not improve for N epochs |
| proximal-mu | 0.1 | FedProx proximal term weight. 0.0 = plain FedAvg. Higher = clients stay closer to global model |
| noise-multiplier | 0.5 | DP Gaussian noise multiplier. 0.0 = no DP. Higher = stronger privacy, less accuracy |
Testing
Install test dependencies and run:
pip install -e ".[test]" pytest tests/ -v
The app validates data and run config on startup. See task.py for validate_npt_csv and validate_run_config.
Publishing on Flower Hub
flwr login supergrid flwr app publish .
Your app will be available at https://flower.ai/apps/drillsense/fed-npt-prediction/.
Resources
- 3W dataset: github.com/petrobras/3W
- Flower website: flower.ai
- Documentation: flower.ai/docs
- Flower on GitHub: github.com/adap/flower
- Flower Slack · Flower Discuss
License
Apache-2.0.