Autoreload/Restart modwsgi on source change

Posted: June 20th, 2010 | Author: Davo | Filed under: Django | Tags: | 3 Comments »

When developing with Django’s DEV server it has a very nice feature, the auto-reload.
That is, every time you make changes to a file, the server is automatically restarted.

We can have the same with Apache/modwsgi, all you need to do is the following:
pre requirements: Linux, python, apache, modwsgi running with WSGIDaemonProcess

1) Find the site-packages of your current python

python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()"

2) monitor.py , copy this file inside your site-packages directory

import os
import sys
import time
import signal
import threading
import atexit
import Queue

_interval = 1.0
_times = {}
_files = []

_running = False
_queue = Queue.Queue()
_lock = threading.Lock()

def _restart(path):
    _queue.put(True)
    prefix = 'monitor (pid=%d):' % os.getpid()
    print >> sys.stderr, '%s Change detected to \'%s\'.' % (prefix, path)
    print >> sys.stderr, '%s Triggering process restart.' % prefix
    os.kill(os.getpid(), signal.SIGINT)

def _modified(path):
    try:
        # If path doesn't denote a file and were previously
        # tracking it, then it has been removed or the file type
        # has changed so force a restart. If not previously
        # tracking the file then we can ignore it as probably
        # pseudo reference such as when file extracted from a
        # collection of modules contained in a zip file.

        if not os.path.isfile(path):
            return path in _times

        # Check for when file last modified.

        mtime = os.stat(path).st_mtime
        if path not in _times:
            _times[path] = mtime

        # Force restart when modification time has changed, even
        # if time now older, as that could indicate older file
        # has been restored.

        if mtime != _times[path]:
            return True
    except:
        # If any exception occured, likely that file has been
        # been removed just before stat(), so force a restart.

        return True

    return False

def _monitor():
    while 1:
        # Check modification times on all files in sys.modules.

        for module in sys.modules.values():
            if not hasattr(module, '__file__'):
                continue
            path = getattr(module, '__file__')
            if not path:
                continue
            if os.path.splitext(path)[1] in ['.pyc', '.pyo', '.pyd']:
                path = path[:-1]
            if _modified(path):
                return _restart(path)

        # Check modification times on files which have
        # specifically been registered for monitoring.

        for path in _files:
            if _modified(path):
                return _restart(path)

        # Go to sleep for specified interval.

        try:
            return _queue.get(timeout=_interval)
        except:
            pass

_thread = threading.Thread(target=_monitor)
_thread.setDaemon(True)

def _exiting():
    try:
        _queue.put(True)
    except:
        pass
    _thread.join()

atexit.register(_exiting)

def track(path):
    if not path in _files:
        _files.append(path)

def start(interval=1.0):
    global _interval
    if interval < _interval:
        _interval = interval

    global _running
    _lock.acquire()
    if not _running:
        prefix = 'monitor (pid=%d):' % os.getpid()
        print >> sys.stderr, '%s Starting change monitor.' % prefix
        _running = True
        _thread.start()
    _lock.release()

3) Finally edit your application WSGI file and add the following before your application=django.core.handlers.wsgi.WSGIHandler():

...
import monitor

monitor.start(interval=5.0) # check every 5 seconds

for root,dirs,files in os.walk('/some/folder/where/your/app/is'):
        for x in files:
            if os.path.splitext(x)[1].lower() == '.py': # only .py files please.
                monitor.track(root + x)
...

original code is at http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Monitoring_For_Code_Changes


3 Comments on “Autoreload/Restart modwsgi on source change”

  1. 1 Graham Dumpleton said at 12:15 pm on June 20th, 2010:

    If you are going to take code out of some one else’s documentation then at least link back to the original with an attribution. I cant see that you have done that. The original code is at ‘http://code.google.com/p/modwsgi/wiki/ReloadingSourceCode#Monitoring_For_Code_Changes’.

    Also be aware that you are using it wrongly anyway. You don’t need to call track() for all files as the code will automatically track changes related to any loaded Python code files. You only need to call track on non Python code files which a Python framework might not itself auto detect changes in. In other words, most web frameworks detect changes automatically in page templates and will reload them internally. Thus pointless having this monitor code checking those.

  2. 2 Davo said at 12:24 pm on June 20th, 2010:

    Hello Graham,

    I updated the post so it links to modwsgi.

    I don’t get what you mean by using it wrong and tracking on NON PYTHON code?

    I am using Django here and monitoring only the .py files isnt this the correct way of handling it ?

    OUT OF TOPIC:
    I still have the issue killing the processed spawned by modwsgi process, remember the ones which become parent when its restarted? :)

  3. 3 Graham Dumpleton said at 12:31 pm on June 20th, 2010:

    The code already tracks the .py files, you don’t need to call track() for them explicitly. It knows which .py files to monitor as it scans through sys.modules looking for what modules have actually been imported by the application and what the corresponding .py files for them may be that need to be checked. Check closely what the loop in the code over sys.modules.values() does.


Leave a Reply