JD index text format - ideas and comments, please

Also notice you’re using colons in your numbers there. Unless there’s a strong argument for — which I’m very happy to consider — my current view is that this is unnecessary and not allowed.

Given the fact that each number is highly structured, we don’t need that divider to help a program figure out what’s what.

Perhaps, but we’d have to re-structure your example so that 200-299 appears in order. Unless what you’re after is some sort of table of contents right at the top of the file? Which might be an interesting idea.

Another thought: break up a large multi-project file, so split each project out to a separate file. But then allow the one master file to include those sub-files.

Adds complexity, might hold that thought for a while.

Funny, I was actually always putting the ID in my index. Maybe that was why I have been finding it so tricky getting started! I’d have to have another go and see what a difference it makes.

Make sure to stipulate the type and quantity of whitespace indentation!

I actually thought that I had removed them. I agree to their being no need for the colons.

Yes I was aiming for a table of contents, also inside the project or area you can have further TOCs.

I’m not sold on this one yet.

The user, or any particular program, may choose to put them in themselves. But they don’t add any semantic value — none whatsoever — so I don’t see any benefit in making them mandatory.

Because I can see two different people preferring two different whitespace schemes:

10-19 Area
   10 Category
   10.00 Option A, where you line up with the number

vs.

10-19 Area
   10 Category
      10.00 Option B, where you line up with the text

Both should be equally valid. Both represent exactly the same data.

The numbers are what makes the file valid or not. The numbers are all a parser needs to determine validity. My system has this built-in advantage (vs., say, YAML or whatever). I don’t think it’s sensible to build in a restriction for the sake of it.

Happy to hear the counter-argument!

Interesting, will need to flesh this one out a bit.

For now of course you can put whatever you like in a comment block at the top.

I’m a bit confused by the effort on creating an index that will store location information. Why are your documents and info stored in multiple platforms that don’t allow cross-platform search? At least on Windows, there are a number of tools that search across many types of searchable information, and if there really is some type of info or document that is within a walled search garden, I’m much happier to store it as a link or download a copy as a local file.

I don’t use Pinterist, but most social media has links to posts that I can store in a text file, and I can search all of my cloud locations and local drives through one interface. Am I misunderstanding the need you are trying to meet with “location” info?

i’m not OP so can’t answer for their use case. But i happen to be building a Notion DB that only store the JD index items with description and location. I use JD to organise my whole life (sure is fascinating to explore my own life condensed in 9 * 9 * 99 items, but i digress…). they live on different places:

  • file system (laptop; phone; external HDD)
  • various cloud file storage services
  • specific online services (time trackers, calendars, expenses tracker, etc)
  • physical (certificates; books and music collection)

Example:

  • 19.01 is my legal IDs (birth certificate, national identification, and passport). I can quickly view the location of the physical documents and know that the scans are in my Bitwarden encrypted vault.

(note: it is indeed possible to sync all cloud services into one central device, but i personally don’t need that capability and don’t feel like spending much money for the hardware and subscription costs.)

having just the index and the surrounding context makes it easier for me to know what i need instantly.

Replying to both @ekafyi & @cobblepot.

Thanks for both your messages. I’m beginning to realise I got off on the wrong foot with JD. I know the power is there to get things literally sorted but I think I went off on moving directories around first.
:man_facepalming:

@ekafyi you made me realise more than ever that the index is about pointers rather than the literal item. I expect that eventually I would move some things around to make the index simpler. But not the other way around. Notion is a good idea. I would use that.

I’m probably still going to go down the route of writing a program to assist. Perhaps a web-based opinionated JD index editor would be a good thing to have?

@cobblepot I am keen to avoid becoming dependant on a cloud storage or running large storage at home. My aim is to be able to manage multiple file systems which have different use cases and volumes. If you could share the name of the one interface that you are using, I may have missed a trick! I currently use Google Drive and couple of computers. I’m interested in using the different machines together to achieve backups. But I think this really is a different problem now.

What @efakyi has said in their reply is closer to the absolute truth which is that JD is about knowing where things are rather than being where they are.

@johnnydecimal I’m still here and thinking about it all. But very busy at the moment. Ironically, I would not be this busy if I had an effective JD index in place!

Now if I can just find my mortgage documents…

@ekafyi nailed it, yeah. Here’s another very specific use-case just for more context. This one is my real-life.

You’re at work. You’re managing a project. One of the categories is 72 Cabling in a data centre.

For some of the IDs in there you’ve got files that you’ve saved in your file system:

72.01 A thing
72.02 Another thing

Then an issue comes along and it only exists in your email. It’s a proper issue, something you need to track, something you want filed in its own place, as you should. So you check your file system, you see that 72.03 is the next number, you use it in your email, happy days.

… later that week …

It’s hectic at work and you get another thing that you need to save in 72. You check the file system. What’s the next free number? 72.03. Great, go ahead and use it. And now you’ve got two things with the same number, and life will never be the same again. The birdsong stops. Your coffee is always just a bit too cold. They announce the end of NCIS re-runs. Sadness prevails.

You need an index. It’s a pain in the ass, and I wish you didn’t, but experience has shown me again and again that you do.

When I re-do the JD site some time later this year — based on everything I’ve learned from this fine forum, thank you all — this sentiment will be at the very top. This is absolutely it, and if you have this front of mind while you “do JD” you’ll do well.

In my view, if you use 72.03 but fail to record its meaning by immediately making a folder for it, that’s clear user error.

I wasn’t questioning the value of an index, I was wondering about the need for a “location” field. I think part of the confusion is that I think of JD as a filing/reference/storage system more than a project-management system. I have a separate system for task and project management, and I don’t use JD to refer to physical items, so basically for me JD is a group of file folders and anything outside those folders (like a web site or something) is recorded in a text file within those folders. The folder structure itself is/acts as an index for me, and I get that the structure could be abstracted into a pure text form that would be a slightly different type of index, but recording location information for each item seemed like a lot of overhead to me.

I store notes and documents in my JD folders, use “Everything” (Windows) as a super-fast file and folder search, and locate anything I need in seconds. The only things I have trouble finding are non-digitized records, and hopefully I’ll get the needed ones digitized soon enough.

1 Like

But why would I make a folder in my file system when I don’t have any files to keep?

Unless the file system is my index, which doesn’t feel neat to me. Now I might go to that folder and wonder why there’s nothing in it. I shouldn’t have to remember that the stuff is in my email.

With an index, I remember there was “that job about the cabling”, I check my index, find the job, it tells me that the files are in my email, I go and look at them.

Right, and if you do this then you probably don’t need an index.

And we’re avoiding the case where you think you’re going crazy because you know you did some work related to that cabling thing, but then you go to your file system and you can’t find it. At work this can sometimes be months, years later. You’ll never remember that it was in your email.

“anything outside those folders (like a web site or something) is recorded in a text file within those folders. The folder structure itself is/acts as an index for me”

I like this method, Hook provides the ability to easily make something like this happen on the desktop especially with the Hook to New feature where it creates a new file from the hooked item. On devices its a bit trickier to implement this. My renewed interest in centralizing stems from: As a multiple browser user I struggled in the past with using bookmark sync apps to synchronize bookmarks across browsers and the most recent issue which broke the sync again was the last straw… I’m now using the Hook to New feature to save the browser bookmarks to the folder structure. I’m going to do the same for all reference pieces going forward. Hopefully I can find a way to do this on IOS as well.

1 Like

Let us know if you find a way to implement your hook-like system on iOS :slight_smile:

1 Like

After futzing w/ Notion I am beginning to think that, for JD, it is truly limited to an index function, although I’m unclear how to do the index and keep it up to date… For my files and docs here are the limitations I’ve been hitting:

  • Importing is a 2-step process - import, then move the import to its proper location.
  • Embedding documents provides a preview, but to open the whole document opens a new browser tab and takes you to Amazon Web Services, Notion’s backend.
  • Doing this with spreadsheets and some other formats opens them as a binary file on AWS, not in the creating application
  • Same for anything imported as an attachment vs. an embed
  • Some documents are recognized by the browser, which prompts you to download and open them. But this is still a copy coming off of AWS, not the local copy in your file system

So after working on this for a bit it’s really not good for much.

Unrelated to JD I have moved my project/task tracking to GitHub, my journaling to DayOne, my book tracking/reviews to LibraryThing. Most everything else is better handled in my Finder or ForkLift.

One reason I experimented w/ Notion was that the syncing is seamless, providing a form of online backup. But it’s better to do that w/ SyncThing (to my Mac Mini) and then Google Drive or any other cloud provider vs. Notion.

Just my two cents on this, when coding https://forum.johnnydecimal.com/t/scheme-module-for-johnny-decimal/ I created a grammar to gather data from an index file. The grammar is defined in the jd.meta module and roughly works as below:

AC Title
  AC.ID Label
  | Short description in one line only
  | Long description
  | that may span
  | multiple lines

I don’t support projects (as in PRO.AC.ID) right now, but that could be easily implemented. Also, contrarily to the guidelines the AC headers are just multiples of 10, never X0-X9.

Technically speaking, this grammar is indent-based, thus context dependant, but an easy trick when preprocessing the index file make it context-independent thus easy to parse.

1 Like