Putting files inside an Obsidian vault along with the JDex

10-19 Concepts 14 Build your system
1

I’m lazy and struggle to maintain a consistent folder structure and separate index (JDex!). I really want notes and files in the same hierarchy. To that end, I’ve been toying with the idea of just keeping everything in one giant Obsidian vault.

I wrote a little about it after my first attempt, here: Obsidian and Johnny Decimal | Baty.net.

The Folder Notes plugin is the key, I think. It makes the folder hierarchy behave like it’s a heirachy of notes.

I’m re-testing this using the Life Admin starter and feeling cautiously optimistic about J.D “sticking” this time, but before I get carried away, do we think that the concept is at least superficially sound? This is going to take a while, so please wave me off if you think it’s a bad idea.

2 Likes
2

Hi Jack :slight_smile: you are not lazy, you just haven’t formed a consistent habit in (understandably) keeping up with the large volume of digital files that come your way! I don’t think we are ever “finished” and the system will never be “complete” because you’ll always have new things coming in and old things going out. That is what your inboxes are for. I am a huge fan of GTD by David Allen and would recommend reading his book if you haven’t, it is easy to follow and pretty solid. For anybody who doesn’t know, He uses a weekly review to get clear; process all loose ends, get current: make sure all items are up to date. and get creative: come up with new ideas to improve life and work. His method of capturing is similar to the JD system in writing everything you can think of on sticky notes and taking a few days to really organise your system. It is very good for the practical and “mental” load of being consistent with your second brain, whilst JD is very good for organisation of digital files/second brain in whatever format they come. The JD system isn’t designed to form habits in keeping up with the system, but to compliment those habits I would say? I am looking forward to seeing what else @johnnydecimal and @LucyDecimal come up with next.

I am a huge fan of the new JD life admin pack and have found it to be the best solution to my digital organising problem. There is this feeling of incompleteness because there is so much else of my digital life that really needs different JD life admin versions besides the regular admin, (for example, small bookkeeping business; Johnny is currently creating a small business admin pack) but my gosh it has been a great start and the JD life admin pack has done all of the hard work for me. It was very easy to transfer everything over from Evernote to Apple Notes at the same time as adding the JD life admin to my system. It is very well thought through, nothing is forgotten and it seems too simple to be true lol.

(I do not use Obsidian, I used Evernote and recently switched to Apple Notes so I am not familiar with how Obsidian works, but I am satisfied with the hierarchy in my systems).

3

Thanks for the thoughts. I could probably develop a habit of keeping the indexes synced, but I’d kind of rather not have to, which is why I’m trying to shoehorn Obsidian into doing what I want. I don’t love Obsidian, but it’s the thing I’ve found that’s best suited for testing this keep-everything-together-at-all costs theory.

I often feel like I’m “doing it wrong”, but the combination of the life admin pack and a more relaxed position on subfolders has convinced me to give J.D another go.

4

@jack.baty , I think I’m in the same situation. It just seems to make sense to mix my files and index files, given the tools I have. In my case, using standard Unix/Linux tools, ripgrep and fzf in combination with a file manager that jumps straight to the search result.
My convention is the following tree structure: 10-19 Home/ > 11 House/ > 11.12 Components/ > 11.12_index.md where everything is a folder except the final index.md.
I’ve been thinking about this question a lot in the last few days, and was meaning to ask on the forum if anyone could offer some reasons to stick with a separate index. These are the technical objections I’ve thought of so far, with my counters:

  • As you mention in your blog post, all the other files in the tree will pollute your search results. → I can add filters as I’m searching to e.g. search only in index.md files.[1]
  • you don’t have a clear overview of ‘just the index’ → my specific tooling lets me see an ‘index’ view trivially. I.e. with a keystroke, I can limit my search to only index.md files at depth 3. I don’t know how this would be in Obsidian.
  • You don’t have a single source of truth. You might get confused as you’re navigating your file system. → if viewing the filesystem becomes so easy through a good file manager (or via Obsidian’s interface), then the folder structure can be your source of truth. A search/filter shows me which IDs do or do not exist.

Besides this type of objection, there might be the psychological effect of ‘maintaining the index’. Which @Jayde20 alludes to. Habit forming. I’m seriously wondering if I’m missing this, but when I’m maintaining my Jdex file tree, it feels like I’m working on ‘the official version’.

My interim conclusion: certain tools make it possible for the filesystem hierarchy to function as the index. In the absence of those tools, you have all the disadvantages/pitfalls which make it a bad idea. So I get why the ‘universal recommendation’ is to keep a separate index. And I will proceed cautiously, on the lookout for signs that I’m treading into dangerous territory.

So if your setup gives you the same sense that you are working with the single source of truth with regard to your system, I would say it’s fine.

  1. Also, I can filter out whole swaths of files based on regular patterns in the filenames. I’ve put the Johnny Decimal Workbook and QSPs under 90 Knowledge/99_PKM. But now I have someone else’s JD index showing up in my search! So I put IGN for ‘ignore’ in the folder name, and I can exclude it. ↩︎

5

edit @jack.baty:
writing my reply, I had you/general audience in mind as a graphical desktop user. But I went off and browsed your (delightful) blog, and I see you are experienced at the unix command line. Also, your article about the gravitational pull of Emacs is HIGHLY pertinent. I understand better your motivation trying to get Obsidian to work. That it’s not out of inability to use more configurable tools, but precisely to avoid having to configure the tools …

In that light, I thought I should expand a tiny bit more on my setup as it relates to your question. Not an answer, but some more context to consider. In my setup, what works really well is browsing the folder-tree-as-index, using find, fzf, ripgrep and ranger. What’s totally missing is good (wiki) linking and navigation. I’m kind of hobbling along with two cilinders not firing at the moment. I have zk set up and I can follow wiki links about half the time, but they seem to break randomly every few days so it’s not a long-term solution. I used to use vimwiki but I want something with a proper AST so I can extend it at some point in the future. I sorely miss note creation automation. Currently, I create the ID folders, the index.md file, and add the title to the file by hand … :face_with_peeking_eye: I haven’t written a script to automate this because I haven’t committed to any one ‘wiki’ app yet, while I evaluate which one seems right for my mental model …

The conundrum seems to be, with Obsidian, you get all this UX free, but then you run into things like you’ve mentioned, where for example you don’t have control over the search results like you would with find (am I right?).

I’ve tried Obsidian three times and ran away screaming three times. I forget why, but since it happened three times I’m not trying again. I think it was the maze of community plugins.

I’ve been learning Emacs in the last two weeks because I finally saw the light regarding org-mode. However, contrary to what Karl Voit says, getting started with Emacs is Simply Not Trivial. I mean, a standard install on Debian gives you a standalone and client/server version which use different init files !!! all well and good, but this is a blocker for an absolute beginner trying to modify the configuration.

So … hopefully not too far off topic. All this to say: I think I understand a bit more of the background of where you’re coming from. And as an expansion of what I said above:

I think there is something important here, regarding simplification. You know what has actually worked for me, breaking the gravitational pull of ‘it should be possible if I tweak things just a bit more’? keeping my index in a paper notebook. Maybe you’re more mature than me, and can live with the shortcomings of a tool like Obsidian and still get on with work … :wink:

2 Likes
6

Forgive me for my lack of coding and cool computer skills, the last comments of this post as it stands today might be helpful for you @jack.baty 22.00.0034: The classes of to-do - #32 by Mentat

3 Likes
7

Thanks a ton for this. If I had my way, I’d stick with Emacs and maintain the JDex in an Org-mode document, and use its built-in org-attach features for related documents. I may still try that, because I simply don’t enjoy using Obsidian beyond the novelty of “it’s something different for a change and it makes doing [thing] easier”. (BTW, I recently tested minimal-emacs.d as a good set of Emacs defaults without getting crazy, if you’re looking)

Your system for keeping things in one place seems perfectly viable. Before any “systems”, I put everything in folders and included a README file for notes and additional info. It was simple, but it was also a mess. I still like the idea though, and Obsidian with the Folder Notes plugin makes it feel like my old method. (assuming that’s a good thing :slight_smile: )

I always start with the idea of keeping it simple. This is one reason I still pay for a Bear subscription. Heck, with Bear I can put many of the important documents right in the note, which is pretty handy. However, I am trying to stick with my rule of keeping all of my notes somewhere in my ~/Documents/Notes folder as individual text/markdown/org-mode files.

Perhaps this topic should be titled “Methods for keeping an index and files in one place” or something.

Thanks for the feedback, it helps!

8

Yes, lots of food for thought, there, thanks!

9

This has put something on my radar. I am going to try this for my genealogy research. Making my whole FamilyHistory folder into a vault. Where notes save at the folder of the person. This is great.

Will report back!

1 Like
10

I’m facing the same question. My answer is NO!—don’t do it! Not with Obsidian. Unless all your stored files are text only, placing everything inside Obsidian will slow it down BIG time.

In my index inside Obsidian I’m placing all the 11.XX folders inside folder 11, 12.XX folders inside folder 12, etc. It’s easier to navigate this way, and if you use the Waypoint plugin it’ll generate a nice list of hierarchical links. I’m on a Mac, and Hookmark is massively useful for providing one-click links to the specific files (usually several folders deep) described in the index.

I’m a bit like you, I think; prone to flip from system to system. This one feels different. I can see me not only sticking with it, but relying on it to keep things in order. (After I change all the ampersands to long-hand, that is! :wink: :joy:)

1 Like
11

When I tried this the first time, I ran into performance issues, but attributed them to indexing everything. When configured to only look at .md files, things got better, although I don’t know how much.

Hookmark can be really handy, but I tend to forget it’s there for some reason. ¯_(ツ)_/¯. I should develop a better habit around it.

Given that I only use Obsidian under duress, I’m now testing some options for using a hierarchy of Org-mode files in Emacs, which is where I’m most comfortable anyway. :grin:.

1 Like
12

About to do some deep-reorg of Genealogy records and was searching for anyone using Hookmark for it. Your idea of index files linked to records/documents is excellent - I have SO. MANY. documents in SO. MANY. folders it is just out of control. Thanks…

1 Like