diff options
Diffstat (limited to 'fimon/README.md')
| -rw-r--r-- | fimon/README.md | 157 |
1 files changed, 157 insertions, 0 deletions
diff --git a/fimon/README.md b/fimon/README.md new file mode 100644 index 0000000..31f5c02 --- /dev/null +++ b/fimon/README.md @@ -0,0 +1,157 @@ +# File Monitor Script + +A Python script that monitors a directory for new files and automatically transfers them to a remote host via SCP. + +## Features + +- **Real-time monitoring**: Uses filesystem events to detect new files immediately +- **Automatic transfer**: Transfers files via SCP to a remote host +- **Retry logic**: Retries failed transfers with configurable delays +- **Error handling**: Moves files that fail all retry attempts to an error folder +- **File archiving**: Successfully transferred files are moved to a `.sent` folder +- **Configurable**: All settings controlled via configuration file +- **Logging**: Comprehensive logging with configurable levels +- **File stability**: Waits for files to finish being written before transfer + +## Prerequisites + +- Python 3.6 or higher +- SSH access to the remote host +- SSH key-based authentication (recommended) + +## Installation + +1. Run the setup script: + ```bash + chmod +x setup.sh + ./setup.sh + ``` + +2. Edit the configuration file `config.ini` with your settings: + ```bash + nano config.ini + ``` + +## Configuration + +Edit `config.ini` with your specific settings: + +```ini +[transfer] +# Directory to monitor for new files +watch_directory = /home/user/upload + +# Remote server settings +remote_host = myserver.com +remote_path = /var/www/uploads +ssh_user = webuser +ssh_key = /home/user/.ssh/id_rsa +ssh_port = 22 + +# Retry settings +max_retries = 3 +retry_delay = 30.0 + +# File handling settings +file_settle_delay = 2.0 + +# Connection settings +connect_timeout = 30 +transfer_timeout = 300 + +# Logging settings +log_level = INFO +log_file = /var/log/file_monitor.log +``` + +### Configuration Options + +- **watch_directory**: Local directory to monitor for new files +- **remote_host**: Hostname or IP of the remote server +- **remote_path**: Destination path on the remote server +- **ssh_user**: Username for SSH connection +- **ssh_key**: Path to SSH private key file +- **ssh_port**: SSH port (default: 22) +- **max_retries**: Number of retry attempts for failed transfers +- **retry_delay**: Seconds to wait between retry attempts +- **file_settle_delay**: Seconds to wait after file creation before transfer +- **connect_timeout**: SSH connection timeout in seconds +- **transfer_timeout**: File transfer timeout in seconds +- **log_level**: Logging level (DEBUG, INFO, WARNING, ERROR) +- **log_file**: Path to log file (optional) + +## Usage + +1. Start the file monitor: + ```bash + python3 file_monitor.py -c config.ini + ``` + +2. The script will: + - Monitor the specified directory for new files + - Wait for files to finish being written + - Transfer files to the remote host via SCP + - Move successful transfers to `.sent/` folder + - Retry failed transfers up to the configured limit + - Move permanently failed files to `.error/` folder + +3. Stop the monitor with `Ctrl+C` + +## Directory Structure + +The script creates the following subdirectories in the watch directory: + +- `.sent/`: Successfully transferred files +- `.error/`: Files that failed all transfer attempts + +## Running as a Service + +To run the file monitor as a systemd service, create `/etc/systemd/system/file-monitor.service`: + +```ini +[Unit] +Description=File Monitor Service +After=network.target + +[Service] +Type=simple +User=your-username +WorkingDirectory=/path/to/scripts +ExecStart=/usr/bin/python3 /path/to/scripts/file_monitor.py -c /path/to/scripts/config.ini +Restart=always +RestartSec=10 + +[Install] +WantedBy=multi-user.target +``` + +Then enable and start the service: + +```bash +sudo systemctl enable file-monitor.service +sudo systemctl start file-monitor.service +``` + +## Troubleshooting + +1. **Permission denied errors**: Ensure the SSH key has correct permissions (600) +2. **Connection refused**: Check if SSH is running on the remote host and port is correct +3. **Files not being detected**: Verify the watch directory path is correct and accessible +4. **Transfer timeouts**: Increase `transfer_timeout` for large files + +## Security Considerations + +- Use SSH key-based authentication instead of passwords +- Restrict SSH key permissions to the specific user and commands needed +- Consider using a dedicated user account for file transfers +- Monitor the `.error/` folder for security issues + +## Logs + +The script logs all activities including: +- File detection events +- Transfer attempts and results +- Error conditions +- Retry attempts + +Log level can be adjusted in the configuration file. |
