Notify of changed files from Windows
Given a folder in Windows, create a program that watches a folder and notify of changed files inside that folder.
For example, let's assume we ask this program to watch the folder called
For the sake of the examples below, let's also assume that there is a file
c:\webroot\static files\wp-content\upload\Plüsch_rosa_Höschen_für_das_Eichhörnchen.jpeg which happens to be inside that folder.
We want to know whenever files inside that folder (
c:\webroot\static files in our example) are created, changed, or deleted, by registering those events inside a directory structure right inside
c:\webroot\static files, as explained below.
The changelist folder
The events are registered as lines inside files with particular names. We call the whole arrangement "a changelist", because at the end of the day, it's just a linear list of changes, sorted from earliest to latest change. However, we want our software to be able to read recent entries in an inexpensive way, independently of the way the folder and its files are exposed to the network.
Thus, changelists are stored in a format amenable for incremental read, distributed across files under a directory with a special name,
__sc_changelist__, below the root of the exposed directory.
In our example, this would result in a directory at
We use the 64 bit representation of the Posix time, and create hierarchical directories
named after the hex representation of the 64-bit word in big-endian order, with the four higher octets abbreviated as a single
0, and where the second smallest
octet is a file instead of a folder, including all the changes that happen in the 256 seconds it covers.
Here is an example: the changes to the filesystem between between Monday, January 22, 2018 1:45:00 PM and Monday, January 22, 2018 1:50:00 PM would be registered by taking the timestamps of those two dates:
converting them to hex and including the padding (to avoid the Year 2038 problem):
0x 00 00 00 00 5a 65 ea dc 0x 00 00 00 00 5a 65 ec 08
We abbreviate the first four octets as a single folder named
0 in the filesystem, and
that results in the following files:
__sc_changelist__ / 0 / 5a / 65 / ea __sc_changelist__ / 0 / 5a / 65 / eb __sc_changelist__ / 0 / 5a / 65 / ec
In the list above, the rightmost group of each line is a file:
Notice that the lowest octet is omitted from the combination of directories and file names.
It's OK if files that otherwise would be empty are missing.
In the files themselves (
ec in the example above),
each the lines uses one of three syntaxes, all of which we describe using a simple grammar:
line ::= change_entry | delete_entry | reset_cmd change_entry ::= '~' TOKEN_TIMESTAMP TOKEN_PATH_LITERAL NEWLINE delete_entry ::= '-' TOKEN_TIMESTAMP TOKEN_PATH_LITERAL NEWLINE reset_cmd ::= 'RESET' TOKEN_TIMESTAMP NEWLINE
- TOKEN_TIMESTAMP is a timestamp in hex representation, so that it's easier to match the filename with the timestamps present inside it. The timestamps should be given in seconds, and expressed in hexadecimal with a resolution of seconds, that is, without truncating it the way it is done with the filename.
- TOKEN_PATH_LITERAL is a valid normalized relative path in Unix notation, i.e. something like
my/file/there, without any
..component, not ending in
/, and with a single
/as separator. The path should be given relative to the folder we are watching. In the example, we would use
- NEWLINE is preferred as the Unix newline, but we added DOS newline parsing so that one is also OK.
- The lines in the file are sorted by the timestamps they contain, so that the creator of the file can simple append entries to it as things happen.
- The file is encoded using UTF-8.
There are two types of entries: one that it is used when a file is created
or changed, and another which is used when a file is deleted.
And one command,
RESET, which can be used by the changelist generator when
it starts running to tell ShimmerCat to invalidate its whole internal store.
So, here are some example contents for the file
used as an example above, assuming a German-Japanese store owner wants to upload
items branded after the anime Banner Tail
using a whimsical mixed alphabet[^1] :
RESET 5a65ec00 ~ 5a65ec02 wp-content/upload/Plüsch_rosa_Höschen_für_das_Eichhörnchen.jpeg ~ 5a65ec02 wp-content/upload/睡眠マット_mat.jpeg