How to Convert Long Word Documents into Markdown for Import into Obsidian

I love Obsidian. It’s a desktop wiki with local storage, meaning the pages in the app are just markdown files on your computer. If you’re running Linux or Ubuntu for Windows, you can do a lot of quick file manipulation by dropping into a command line interface, including creating new files. Have a dozen markdown files about your favorite episodes of One Piece? You can create a folder under the wiki root, drop those files in there, and boom, they’re in your wiki, all ready to be automatically indexed and cross-referenced.

Using this feature, we can quickly pull into the 120K+ words from your novels, split them into chapters, and make them searchable and interlinkable pages in Obsidian. And it’s not that hard, either. It barely took me several tries to get it right!

Thanks for reading Notes From the Author! Subscribe for free to receive new posts and support my work.

Why Do This?

I write novels in an ongoing anthology series that takes place in a shared universe where technology has progressed unchecked. The books share characters, organizations, places, and more between them, which means that anytime I use something from a previous book, I have to cross-reference and make sure I’m not creating a continuity error.

Consider a problem like: Have I used the name Terrance Bottomfeather before?To find out, I’d have to open eight large Word documents and run a search, which is tedious. I’d much rather have the text from all eight novels inside Obsidian and therefore searchable all at once.

I’m sure this a common problem.

Convert Word Documents to Markdown

The first step is taking your .docx file and turning it into a .md file. You can do this in Ubuntu with pandoc. Once you have it installed, you just have to feed it the right arguments:

pandoc -f docx -t markdown vise-manor.docx -o vise-manor.md

You can read up on what the options mean, but basically it’s saying take this docx file and output it to an md file. That’s the official scientific explanation.

You’ll get a huge file as a result:

dverastiqui@Ginsberg:~/tmp$ ll
total 1436
drwxr-xr-x 1 dverastiqui dverastiqui   4096 Sep 29 10:01 ./
drwxr-xr-x 1 dverastiqui dverastiqui   4096 Sep 29 09:59 ../
-rw-r--r-- 1 dverastiqui dverastiqui 615787 Jan 20  2022 vise-manor.docx
-rw-r--r-- 1 dverastiqui dverastiqui 815643 Sep 29 10:01 vise-manor.md

You can use less or head to verify the conversion.

dverastiqui@Ginsberg:~/tmp$ head vise-manor.md
ALSO BY DANIEL VERASTIQUI

*Xronixle*

*Veneer*

*Perion Synthetics*

*Por Vida*

Now, you could drop this file directly into Obsidian, but that’s a bit unwieldy. Since the content is already arranged in chapters, let’s just go ahead and chop it up.

Cut Markdown Files into Chapters

This part calls again for pandoc and some minimal CLI scripting. Essentially, we’re going to be converting the markdown file into an EPUB archive and then unpacking the EPUB to convert the xhtml chapters into markdown. Look, it doesn’t have to make sense. Just know that it works.

You may get some warnings, but you can ignore them, especially this one:

dverastiqui@Ginsberg:~/tmp$ pandoc -f markdown -t epub -o vise-manor.epub vise-manor.md
[WARNING] This document format requires a nonempty <title> element.
  Please specify either 'title' or 'pagetitle' in the metadata,
  e.g. by using --metadata pagetitle="..." on the command line.
  Falling back to 'vise-manor'
[WARNING] This document format requires a nonempty <title> element.
  Please specify either 'title' or 'pagetitle' in the metadata,
  e.g. by using --metadata pagetitle="..." on the command line.
  Falling back to 'vise-manor'

dverastiqui@Ginsberg:~/tmp$ ll
total 1840
drwxr-xr-x 1 dverastiqui dverastiqui   4096 Sep 29 10:09 ./
drwxr-xr-x 1 dverastiqui dverastiqui   4096 Sep 29 09:59 ../
-rw-r--r-- 1 dverastiqui dverastiqui 615787 Jan 20  2022 vise-manor.docx
-rw-r--r-- 1 dverastiqui dverastiqui 412195 Sep 29 10:09 vise-manor.epub
-rw-r--r-- 1 dverastiqui dverastiqui 815643 Sep 29 10:01 vise-manor.md

Now unzip the EPUB archive.

dverastiqui@Ginsberg:~/tmp$ unzip vise-manor.epub
Archive:  vise-manor.epub
 extracting: mimetype
  inflating: META-INF/container.xml
  inflating: META-INF/com.apple.ibooks.display-options.xml
  inflating: EPUB/content.opf
  inflating: EPUB/toc.ncx
  inflating: EPUB/nav.xhtml
  inflating: EPUB/text/title_page.xhtml
  inflating: EPUB/styles/stylesheet1.css
  inflating: EPUB/text/ch001.xhtml
  inflating: EPUB/text/ch002.xhtml
  inflating: EPUB/text/ch003.xhtml
  inflating: EPUB/text/ch004.xhtml
  inflating: EPUB/text/ch005.xhtml
  inflating: EPUB/text/ch006.xhtml
  inflating: EPUB/text/ch007.xhtml
--OUTPUT REMOVED--

Did you see that? Converting to EPUB automatically separated everything into chapters. They’re in XHTML format right now, but we can fix that.

dverastiqui@Ginsberg:~/tmp$ for chapter in EPUB/text/*.xhtml; do pandoc -f html -t markdown -o ${chapter/html/md} ${chapter}; done;

dverastiqui@Ginsberg:~/tmp$ ll EPUB/text/
total 2364
drwxr-xr-x 1 dverastiqui dverastiqui  4096 Sep 29 10:12 ./
drwxr-xr-x 1 dverastiqui dverastiqui  4096 Sep 29 10:10 ../
-rw-r--r-- 1 dverastiqui dverastiqui  2229 Sep 29  2023 ch001.xhtml
-rw-r--r-- 1 dverastiqui dverastiqui  1181 Sep 29 10:12 ch001.xmd
-rw-r--r-- 1 dverastiqui dverastiqui  1036 Sep 29  2023 ch002.xhtml
-rw-r--r-- 1 dverastiqui dverastiqui   305 Sep 29 10:12 ch002.xmd
-rw-r--r-- 1 dverastiqui dverastiqui 14026 Sep 29  2023 ch003.xhtml
-rw-r--r-- 1 dverastiqui dverastiqui 12360 Sep 29 10:12 ch003.xmd
-rw-r--r-- 1 dverastiqui dverastiqui 14818 Sep 29  2023 ch004.xhtml
-rw-r--r-- 1 dverastiqui dverastiqui 13056 Sep 29 10:12 ch004.xmd
--OUTPUT REMOVED--

Unzipping the EPUB got us chapters in XHTML format. Now we have duplicated those into XMD format. The next step is to rename all those files.

dverastiqui@Ginsberg:~/tmp$ for i in EPUB/text/*.xmd; do mv -- "$i" "${i%.xmd}.md"; done

dverastiqui@Ginsberg:~/tmp$ ll EPUB/text/
total 2364
drwxr-xr-x 1 dverastiqui dverastiqui  4096 Sep 29 10:15 ./
drwxr-xr-x 1 dverastiqui dverastiqui  4096 Sep 29 10:10 ../
-rw-r--r-- 1 dverastiqui dverastiqui  1181 Sep 29 10:12 ch001.md
-rw-r--r-- 1 dverastiqui dverastiqui  2229 Sep 29  2023 ch001.xhtml
-rw-r--r-- 1 dverastiqui dverastiqui   305 Sep 29 10:12 ch002.md
-rw-r--r-- 1 dverastiqui dverastiqui  1036 Sep 29  2023 ch002.xhtml
-rw-r--r-- 1 dverastiqui dverastiqui 12360 Sep 29 10:12 ch003.md
-rw-r--r-- 1 dverastiqui dverastiqui 14026 Sep 29  2023 ch003.xhtml
-rw-r--r-- 1 dverastiqui dverastiqui 13056 Sep 29 10:12 ch004.md
-rw-r--r-- 1 dverastiqui dverastiqui 14818 Sep 29  2023 ch004.xhtml

Those MD files are now ready to move into Obsidian. Just make sure you know where you want them and use the mv command.

dverastiqui@Ginsberg:~/tmp$ mv EPUB/text/*.md /od/writing/Obsidian Wiki/Vinestead Universe/zFull Text/vise-manor/

dverastiqui@Ginsberg:/od/writing/Obsidian Wiki/Vinestead Universe/zFull Text$ ll vise-manor/
total 940
drwxrwxrwx 1 dverastiqui dverastiqui   512 Apr 19 13:40 ./
drwxrwxrwx 1 dverastiqui dverastiqui   512 Sep 21 19:37 ../
-rwxrwxrwx 1 dverastiqui dverastiqui  1181 Apr 19 13:40 ch001.md*
-rwxrwxrwx 1 dverastiqui dverastiqui   305 Apr 19 13:40 ch002.md*
-rwxrwxrwx 1 dverastiqui dverastiqui 12360 Apr 19 13:40 ch003.md*
-rwxrwxrwx 1 dverastiqui dverastiqui 13056 Apr 19 13:40 ch004.md*

Now you should be able to open Obsidian and see the new content.

Congrats! You did it! Enjoy searching your previous novels with ease!

References

https://medium.com/geekculture/how-to-easily-convert-word-to-markdown-with-pandoc-4d60878ccc64
https://linuxconfig.org/how-to-rename-multiple-files-on-linux
https://stackoverflow.com/questions/33889814/how-do-i-split-a-markdown-file-into-separate-files-at-the-heading

The Author is Going to Regret This

Drink enough alcohol with me and I’ll let you know exactly how I feel about Goodreads Giveaways. How they cost too much. How they don’t result in reviews. How physical copies end up on Ebay the next week with the description brand new, never opened. It’s just not a good marketing strategy compared to everything else available to indie authors. Still, when you reach the end of the year with a surplus in your marketing budget, it’s easier to accept throwing money away on a giveaway. At least it pumps up those to-read numbers, right? Anyway, I listed a giveaway for Vise Manor.

Goodreads Book Giveaway

Vise Manor

by Daniel Verastiqui

Giveaway ends January 09, 2023.

See the giveaway details at Goodreads.

Enter Giveaway

It has been an interesting year since Vise Manor was published. Sales have been great, and feedback has been good, but reviews have been few and far between. There’s something about visiting my author page on Amazon and seeing the paltry numbers next to each book. What I wouldn’t give to see those numbers multiplied by a hundred or more. Speaking of reviews, here’s what Carl over at smallchangeonstrangers.com thought of Vise Manor…

Daniel Verastiqui’s latest novel at once sticks with what he knows but then also strikes out in an interesting new direction.

Sure, we’re in the Vinestead Universe. Sure, there’s AI, hackers, and body modifications…but now we’re in a claustrophobic, classic locked manor house murder mystery – and the two genres blend together surprisingly well.

Verastiqui juggles multiple characters with apparent ease and keeps the pace moving through the set up, the shocking but inevitable violence, and then the desperate struggles each character has to survive to the end of the night (and the book).

Do I hate him for some of the things he put my favorites through? Yes, yes I do. And I can give no higher praise than that: I cared about these people. I wanted some to thrive and didn’t mind if the machinery of the story ground others to (metallic) dust.

If you like sci-fi, country house murder mysteries, or just want to care about authentic characters in near-constant peril, then I recommend this book.

Aside from the review, Carl sent me a private text with more thoughts on the book, which I really appreciated. While authors love a favorable public review, they always want to hear more in a more personal setting. I, myself, have a contact form you can use to tell me how you really feel about the Vinestead Universe books. All I want for Christmas is some feedback. And an Infiniti Q50 Red Sport.

There is no better Christmas treat than my mom’s pan de polvo

Anyway, I’ve got another book to write and diapers to change and chorin’ to do and Odins to murder and pan de polvo to binge eat. Oh, and my day job. I keep forgetting about that.

The Author Recommends: Das Keyboard Professional 6

Although I’m a big believer in YNAB, there are just some things in life for which you cannot have a budget. One of those is books (obviously), another is Pop Tarts, and the third is anything that makes your job as an author easier. Yearly subscription for Microsoft Word even though it used to be a one-time fee? Sure! Yearly subscription to Novlr even though you only use it for a total of three months every two years? Yes! $200 every couple of years for a new Das Keyboard mechanical keyboard even though your last one still works great? Shut up, and take my money, good sir.

Okay, in all honesty, I skipped the Pro 5 version. I’m not made of money. Plus, the 4 was working so well–why mess with perfection? I’ll tell you why! Because you can! It just so happened I found myself with a little extra coin in my pocket when Das Keyboard released the Professional 6 last month (or September). I ordered it a few days after its release, and since they’re based here in Austin, Texas, I received it pretty quickly. I couldn’t wait to see how it compared to the 4!

Obligatory Photos

One thing I immediately noticed was how much smaller the 6 is. The first picture doesn’t do it justice, but look how they stack up in the second. The bezel is thinner, the flare in the upper right is less pronounced, and somehow it just feels tighter… less… significant. No, that’s not the right word. Ugh, they should have sent a poet.

I suppose what it lacks in height and width, it makes up for in girth. Look at that thick boy 6 (bottom) compared to the 4 (top). Your fingers are definitely sitting a little higher off the desk now.

The world is moving to USB-C, and Das Keyboard is no exception. Although they were kind enough to include an adapter so I can still plug it into my aging PC, I was kinda bummed to lose the two USB ports on the top. They’re USB-C now, and I don’t have anything to plug into them. So they’re just sitting there, unused, like the coleslaw in my fish combo at Long John Silver’s.

Listen, people. I’m a “professional” writer, okay? I don’t go in for those gimmicky LED backlights that coruscate (see?) through the entire color wheel. I don’t even look at the keys, so why would I need them lit up? Honestly, the LEDs are a major reason I skipped the 5. Too busy. Too much. Calm down, keyboard!

That said, I like the subtle glow of the keyboard now. It makes it feel more “alive” on my desk without being distracting. A button lets you adjust the brightness, which is appreciated.

Initial Impressions

The 6 I ordered used the same Cherry MX Brown switches I have on my 4, so the typing experience was more or less the same. There was no loss of speed or accuracy. There’s a slight tactile difference to the keys (same as between the Pro 4 and Ultimate 4 that I use at work). It types fast as hell and I love it.

I may be the only one who can hear it, but there is a definite metallic squink present during typing if you are in a very quiet room, audible even over the clacking of the keys. I imagine it has something to do with the metal clasps under the bigger keys, but who knows. It’s not enough to make me complain, and I always have some music going anyway.

After a couple of days, I realized one of the Windows keys wasn’t working. Thinking I had a defective keyboard, I wrote into their support. They were helpful and informed me I would need to flash the firmware. That’s all well and good, but this is the app they told me to download:

Not for nothin’, but I get real skittish about clicking buttons I can’t read. It did resolve my problem though, so that’s a plus.

Final Thoughts

I’ve been using the Das Keyboard Professional 6 for about a month now while working on a new book. I can’t say there is a huge improvement over the 4, but it is definitely the kind of rock-solid mechanical keyboard you’d expect from this company. Ultimately, I consider it a good purchase, not only because I have a new shiny keyboard, but also because I get to support an Austin company that I’d definitely work at if they offered me a job. I will test keyboards until my fingers fall off if that’s what it takes!

If you’ve got a Pro 4 or 5, you could consider skipping this upgrade unless you’re swimming in money from all your Kindle Unlimited reads. If you’re 3 or earlier, it is definitely time to upgrade.

I don’t know how authors write without a mechanical keyboard, and I don’t know how I’ll ever write without a Das Keyboard. A+++ would buy again.

Visit them at daskeyboard.com.

I’m Kinda Over Kindle

A couple years ago, I made the biggest mistake of my life: I got rid of all (most) of my print books in favor of Kindle. What can I say? I was taken in by that minimalism doc on Netflix. Fast forward to today, and I lament not having a bookshelf overflowing with science fiction and mysteries and light erotica–you know, all the genres my dad kept on his bookshelf when I was a wee little cyberpunk writer. Although we moved every 4-5 years, that bookshelf followed us from deployment to deployment, and always stood a post in a shared space like a hallway.

I loved that bookshelf. It’s where I discovered Replay by Ken Grimwood. Almost every book was beyond my comprehension, but I read them all the same. My son, El Matador, can read, and he’s not even five years old yet. He has tons of books in his room, but I can imagine him one day coming into my home office, standing in front of my bookshelf, and pulling out a copy of Neuromancer, The Martian Chronicles, or A Scanner Darkly. Sure, he might grab the Henny Millers or Anaïs Nins, but them’s the breaks. At least that’ll give him the foundation to grow up and write stories like his old man.

Anyway, I’m no longer buying any Kindle books, no matter how deeply Amazon discounts them. The irony is I still rely on Kindle sales and Kindle Unlimited reads for my books. In the long run, I want that bookshelf in my house full of awesome books that will inspire my children. As a kid, I dreamt of what my library would look like when I was older–some sprawling room with floor-to-ceiling shelves.

Instead, it’s just a thin plastic pad that shows me ads between the books I actually want to read.

We’ve strayed too far, my friends.

The Lifecycle of Independent Publishing as I Understand It (2021 Edition)

If there’s one thing in this world that I truly love, it’s process. Process is what gives life meaning. It tells me what to do, how to do it, and when to do it. Unfortunately, process does not spring spontaneously from the ether (nor do you get it from a single blog post that claims to teach you How to Write a Novel in 5 Easy Steps); it must be refined over the years. After independently publishing NINE books since 2004, I’m still updating the way I put out books.

Here is my current process, pared down to the bare essentials.

Step 1: Write the Book

What fun! This step is full of late nights and endless glasses of alcohol. Creativity flows from your fingers like human waste from a broken sewer pipe. You create a world, populate it with flawed characters, and watch the chaos ensue. 150,000 words and many, many revisions later, you have yourself a book.

Cost: Time, blood, sweat, tears, proper liver function

Step 2: Pay Someone to Edit/Proof Your Book

You can’t trust yourself or your canine writing partner to edit your book, so you need to hire a professional to look over your work of art. Hope you had some money saved, because good editors and proofers cost a pretty penny.

Cost: $500 – $2,500

Step 3: Pay Someone to Design a Cover

The last thing you want is your book appearing on Amazon with a cover that looks like you designed it yourself. Don’t judge a book by its cover is terrible advice when it comes to marketing your book. Hire a pro. Just hire a pro.

Cost: $200 – $500

Step 4: Pay Amazon to Advertise It

I’ve only found one method of advertising that produces results, and that’s Amazon’s own advertising platform. Create your ads, set your keywords, and give it a healthy budget. Watch the sales and Kindle Unlimited reads come pouring in!

Cost: $300 – $1,000 monthly

Step 5: Use Royalties to Pay for Ongoing Advertising / PROMOTIONS

Did you earn less in royalties than you spent in advertising? If so, you’ll need to pay yourself back first. Isn’t that funny? You put a book on Amazon, pay Amazon to advertise it, and then give all the money you earn back to Amazon to continue advertising it! It’s a vicious cycle, but fingers crossed you can break out of it one day.

Cost: Everything you earned

Step 6: Go Back to Step 1

Now you’re done. Or at least, I am. As you can see, my process doesn’t include any extraneous steps like:

Querying the manuscript
Advertising on Facebook / Twitter / Pinterest / Tik-Tok / AppChat / Fartbook
Advertising on mailing lists or daily deals sites
Advertising on Bookbub
Running giveaways on Goodreads so winners can resell the books on eBay
Maintaining a social media presence of any kind

It’s a simple process.

Create product
Advertise product
Sell product
Repeat

For a Limited Time Only

I suppose it comes down to what you’re willing to do to make your dreams of becoming a best-selling Science Fiction author come true… and how much time you have to commit to it. I make time for writing. Before exercising, the full-time job, and sometimes even my family, I make time for writing. The actual putting of words on the page is #1, but everything else like marketing and social media are so low on the priority list that I’m surprised I even get to them.

After almost twenty years, I think I’ve reached the point where I’ve decided this is how much I’m going to do, and that’s it. In some ways, it’s freeing. I know exactly what I’m doing with my stories, which steps to follow, and when to start over again. My newsletter may not have thousands of subscribers, my blog millions of visitors, or my Facebook fan page billions of Likes… but there will be dozens of my novels out there by the time they put me in the dirt or transfer me to a synthetic body.

And that means I get to write those novels, which honestly, is the only part of this racket that I enjoy.

Louis Sachar and the Literary IDGAF

For the first three years of his life, my son has enjoyed classic books like Goodnight Moon, The Pout-Pout Fish, and The Going to Bed Book. More recently, I’ve been trying to expose him to more mature books like The Ghost on Saturday Night, My Teacher Fried My Brains, and of course, the absolute best children’s book in history, Sideways Stories From Wayside School by Louis Sachar. I can usually get a page or two into those before he complains how long is this story? And when I calmly explain to him how much of an impact Sideways had on me as a child and an adult, he pretends to get tired and fall asleep. Such is parenthood.

Continue reading →

A Month with Novlr

NaNoWriMo is coming up soon, which means a new crop of writers will be looking for the right software to help write their 50,000-word opus. Enter Novlr, a web-based writing platform that is simple, clean, and eager to motivate you to reach your goal. If you’re doing NaNoWriMo for the first time, Novlr is a great choice. But what about the rest of us? What about the seasoned amateurs who haven’t done NaNo (as the kids called it back then) since they were tiny little dystopian cyber-thriller writers? Is Novlr a good choice for writing novls? Let’s take a look.