Jdex out of sync with filesystem

My question: is it normal/expected at first to have problems keeping jdex and filesystem in sync? I wonder if I’m making a fundamental error that’s making syncing difficult.

Details:

I’m new to jd and I keep track of my work projects in a jd system.

At first, I created an elaborate folder structure with a hierarchy like 20-29 Work > 22.10 ⯈ Bugs Tickets Requests :warning: > 22.11 BugReports > index.org, where index.org is 4 levels deep and it’s where my actual notes, links, etc would go for bug reports.

But my system will be in flux for a while because I keep finding better ways to organize items. This instantly turned into a maintenance nightmare as I tried to keep a changing jdex file in sync with the changing filesystem organization.

I could generate a jdex file from the filesystem layout, but every time it’s regenerated I’d lose any notes in it.

Now, to keep things simple, I’ve doing without a jdex file and keeping a flat folder/file system with every item at the first level of the Work directory:

/Users/billw/jd/20-29.Work
[...]
├── 22.10 ⯈ Bugs Tickets Requests ⚠️
├── 22.11.BugReports.org
├── 22.11.BugReports.org_archive
├── 22.12.JiraTickets.org
├── 22.13.OtherRequests.org
├── 22.13.OtherRequests.org_archive
[...]

Hi.

Very, very recognizable. This is one reason why I keep on wanting to keep my ‘jdex’ spread throughout the filesystem: with an index.md – or index.org or whatever – in the folder of each ID.

The following is my attempt at generalizing your question, since it resonates very much with my yoyo experience.

The canonical Johnny Decimal way is to keep notes in the JDex. I.e. in the notes stored under 00.00, while the rest of the JD filesystem is for ‘bigger’ files. But I think your example of bug reports stretches the limits of this approach. It seems like you’re a Plain Text kind of person, using org files to track bug tickets.

I wonder if the following either/or question applies:

a) for plain-text-everything people, it needs to become acceptable to admit that a different approach/format to the JD system applies. Use index.* files through the filesystem, find them with whatever fast file search you use, and don’t look back. The separate-JDex format is for the majority of people out there who see their notes as records in an app, and have less need for plain text notes, and do all their work in various apps for which (binary) filetypes are the logical artifacts. Think Word documents, Spreadsheets, PDFs, audio/video, or an external issue tracker like Github.

b) there is still value in maintaining a separate JDex, even for plain-text-everything approaches. It allows you to plan your structure separately from the implementation. This is a collection of plain text files at 00.00, but see this more as metadata and put notes there accordingly. So in the case of bug tracking, this would only contain something like ‘I track bugs for my own software in org files at 22.10; bugs for contract work under agency X in Jira.agency-x.org under account Y, and bugs for contributions to open source project Z at Codeberg/Z …’ I.e. this is your lookup table to make unambiguous where you do what work, but not where you do the work itself.

This would allow a single 22.10 Jdex file which you could update as you modified your file structure, without having to change the Jdex structure. It requires a clear separation of what kind of notes you keep in each, or in other words, having such a separation needs to make sense to you. This would possibly eliminate a bunch of the IDs in your example: all external bug trackers like Jira for Agency X could be listed here (and delisted when no longer relevant, e.g. moved to 00.00/jdex.org_archive) and wouldn’t need a separate ID. If you track some bug requests in org files in your filesystem, then it would suffice (in my mind) to have a note in your JDex saying so, and have perhaps just 22.10.BugReports.org, 22.10.OtherRequests.org

Another way of putting this is that the JDex structure would emerge from a post-it-on-whiteboard data collection process. Designing a JDex is more about collecting everything and then creating a hierarchy which unambiguously categorizes things into two levels. It’s really implementation-independent. I wonder if the problem arises for us because we work the other way round, and we have the unfortunate coincidence that the JDex is plain text and so is the rest of our day-to-day work. For many people that’s not the case (even if you use Obsidian, which stores things in plain text, the experience is that the notes are collected in an app, separate from The Actual Work. Org-mode or Vimwiki give the experience that notes are intermixed with The Actual Work.)

Maybe you’ve already cycled through the option I explored above! This is my thinking aloud using your example to iterate a bit … I hope that helps at all, but in any case it’s a recognizable problem for me.

1 Like

I was just thinking on this earlier, and wishing I’d written a minimal CLI utility that sat in the background and monitored your system & JDex for inconsistencies. Just a simple scan of both, and some sort of alert if they diverge.

Of course that’s easily said and harder done due to the sheer variety of setups out there, but I’ll add it to a list of things…

@hans - thank you for your thoughtful answer.

It seems useful to distinguish between a jdex that points to work done “out there”, and one that becomes part of the work done. My jd hierarchy is literally where I do most of my work each day. When I was setting this up at first, it was just obvious to me without a second thought that my work would go into the hierarchy directories.

Perhaps I adopted & moved into an Official Hierarchy too soon. I seem to be in the whiteboard phase still, figuring out where things go. So it may be unwise to just move everything into the jd dirs this early when my system isn’t even settled and debugged yet. At this point, when I rename a jd element or move it to a different dir, my actual work files are moving.

I should probably freeze the dir structure and spend some time crafting a jd hierarchy in a real whiteboard phase - just as @johnnydecimal recommended! :slight_smile:

Maybe working inside the hierarchy is a bad idea, and the jdex system should refer to those work products over there in an abstract sort of way, not literally these files inside the jd system.

I’ll re-read the manual with these distinctions in mind and see what jumps out.

Basically, it’s all in the Workbook. I skipped ahead to the fun stuff without doing the work that @johnnydecimal found to be necessary.

2 Likes