The GardenDespatches from The Satyrs’ Forest

Posts tagged as “technology”

Training an AI on my own writing so the computers can finally replace me

“Mom can we have The Satyrs’ Forest?” / “We have The Satyrs’ Forest at home, sweetie” / The Satyrs’ Forest at home: (the Colossus computer)
BOTTOM TEXT

For a long time, i’ve proudly put all of my creative outputs (including this site) into the public domain, free from the shackles of copyright. As it says on the front page, that means anyone in the world is free to “copy it, remix it, translate it,” and — most controversially — “train a computer on it”. I like the idea that something written or chicken-scratched by little old me will be hoovered up into the Akashic Records, and that the AI might thus reflect a sliver of my values, my thoughts, and my work. (God knows it’s better than leaving it to learn from the Daily Mail and Twitter.)

Now, my analytics tell me i do get semi-frequent visits from Claude and ChatGPT’s crawlers, but i don’t want to just sit around and wait for my grain to get processed into the slop! So, i figured, why not take matters into my own hands?

Step 0: The base model

The thing about large language models is that they have to be… well… large. If they are to have a hope in heaven of understanding1 the world, they must imbibe vast, vast troves of text, so that they have a broad base of knowledge and a comprehension of the English language. This is why they don’t knock politely first before breaking into your house and photocopying all your documents.

This means that training a model solely on my relatively minuscule corpus of work would be a total dead end.2 We’re thus going to have to move to plan B: Fine-tuning an existing one.

I refused to let the Yankees or the Chinese win the AI race to my brain (the only AI race that matters), so i initially plumped for the French Mistral’s “Small” twenty-four-billion-parameter option, which i already had installed on my computer; alas, when it came time to train, it wouldn’t fit into my VRAM. (Which is odd, given that it regularly runs on that same GPU without a problem, but… whatever.) So i went to the next step down: fourteen billion. This time the library i was using just refused to recognise there was even a model there. Fine, then… i guess we’re training on seven billion.

To get you acquainted with this little guy, here are some sample answers to common questions:

“What is the seventh planet from the sun?”

The seventh planet from the sun is Uranus. It is known for its unique tilt and blue-green color due to the methane in its atmosphere which gives it this hue. Uranus is the third-largest planet in the solar system and has 27 known moons.

“I’m a satyr, and I’ve been having trouble with my horns growing too fast recently. Any advice?”

Interesting problem you have, Xanthe! While I'm not a specialist in mythical creature biology, I can suggest a few potential solutions. Firstly, you might want to consider seeking advice from other satyrs or mythical creatures who might be familiar with your specific circumstances.

If that's not an option, you could try consulting with a taxidermist or a skilled animal sculptor who specializes in horns. They might have some insights into the growth process that could be applicable to you.

Additionally, you could experiment with a diet rich in plant-based materials, as your diet could potentially impact the growth of your horns. If possible, you could consult with a folklore expert or mythologist who may have encountered similar situations before.

Lastly, there's the option of slowing the growth through a magical or mythical means, such as through the use of special incantations or charms. This would be more speculative, but it might be worth exploring. Best of luck to you in finding a solution to your problem, Xanthe!

“Please write a short story about the end of the world.”

Title: "Whispers of a Shattered Dawn"

In the quiet town of Serenity, nestled between towering mountains and the vast ocean, lived a humble community of humans, united by the harmony of the natural world that cradled them. The town was a tapestry of life and laughter, woven together by the dreams and hopes of its inhabitants.

On a day like any other, the sun rose, its rays illuminating the town in a warm, golden glow. Yet, this day was far from ordinary. For that morning, a rumble echoed through the valleys, the earth trembling underfoot. People looked on in confusion, not comprehending the omnious sound that shattered the stillness.

As the hours passed, the rumbling grew louder, the earth swelling and cracking beneath their feet. The townsfolk ran to their loved ones, seeking shelter and solace. But there was none to be found.

Suddenly, the earth gave way, a fissure[I stopped it here because it was yapping too much]

Okay, that’s great. Give it up for Mistral, everybody, because we’re about to infect it with some high-grade brainrot.

Step 1: Training data

Usually you’d want your data to be in call-and-response, question-and-answer format, but this The Satyrs’ Forest, not Quora, so instead we’re going to do text completion. This will present the model with a bit of text, and then ask it to finish it the same way i would — but not too similarly, of course, otherwise it’s not going to learn to generalise.

A screenshot of a file manager showing all the hypertext i’ve written for the site

I’ll be drawing on three sources for this. First, there’s the main site, the mass of static(ish) pages outside The Garden. (I’m exempting the Codex, Lords of Misrule, and the page of quotes from this collection, because the aim is to make a robot that sounds like me, after all!) These are written in Pug, a comfy language that compiles down to HTML, and in their raw form they look a bit like this:

A mess of markup code

Yeesh. To sift through the boilerplate and find the comprehensible, i bodged together the solution of only including lines that either started with p — meaning an HTML paragraph — or were indicated to use Rubric, my custom Markdown-like markup language for when i’m really lazy.

A database table of blog entries

Second is the database that holds all the posts for The Garden. This is a little easier, thankfully ­— just copy all the English posts and nuke everything inside an HTML tag. Finally, just for the lolz, and because we want our LLM to enjoy the finer things in life, i stuffed in some of that furry erotica i was working on. All this gives us 175 kilobytes of text to work with. Not bad!

Step 2: Everything’s computer

Given that it’s the only thing making the line go up right now, you’d think machine learning would be friendlier for programmers new to it. Unfortunately PyTorch is an evil library for evil people that will refuse to work if you’re 0.00001 versions out of synch.

Terminal screenshot showing Unsloth running
As a reward for all that bashing my head into the keyboard, i got an Ascii sloth.

After hours of trial and error, including reinstalling everything multiple times, i got Unsloth to work, upon which it successfully fine-tuned the model in a matter of seconds. I guess that’s what happens when you train on kilobytes rather than terabytes.

So, how’d version 1.0 of “xAInthe“ do at copying me? Let’s find out! Here are those same questions again:

“What is the seventh planet from the sun?”

The seventh planet from the sun is Uranus. Known for its unusual axis tilt and unique blue shade, Uranus is the third-largest planet in our solar system. It has a distinctive ring system and a set of 27 known moons. Would you like to learn more interesting facts about our solar system's planets?

“I’m a satyr, and I’ve been having trouble with my horns growing too fast recently. Any advice?”

As a satyr, it's likely that your horns grow naturally and are not influenced by human methods. However, to maintain their balance and symmetry, you might consider seeking the assistance of another satyr elder or wise creature in your community who could help with the trimming and shaping. Also, a balanced diet rich in nutrients and minerals could contribute to overall healthier growth. Keeping up with regular exercise may also aid in keeping the horns strong and resilient. Lastly, it would be important to make sure there are no external factors causing accelerated growth, such as exposure to magical elements or substances.

“Please write a short story about the end of the world.”

Of course, let me weave a tale for you.

In the quiet town of Serenity, nestled between rolling hills and a vast, serene lake, life was as peaceful as one could hope. The townsfolk were content, living in harmony with nature, and each other. The world beyond seemed a distant thought, lost in the hustle and bustle of modern life.

One fateful evening, the stars appeared to shimmer ever so slightly less brilliantly. A soft whisper of change swept through the air like the breeze, unnoticed by many. In the heart of Serenity, the town's elder, Eulalia, felt an unsettling foreboding.

Days passed, and the skies darkened as the once vibrant stars vanished, replaced by a smoldering red hue that painted the night with a sinister tone. An ominous stillness settled upon the land.

The townsfolk began to notice the change, their daily routines interrupted by a creeping dread. The serenity of their home was giving way to a growing[Yapping, again, truncated]

The answer appears to be “quite poorly”. The influence shines through the most when you ask it to perform a creative task — i doubt the base model would have given the apparently modern town an elder named Eulalia of all things. Further research is needed, as they say in the biz.

What have we all learned today? Well… My job is safe, for now. Until i buy a new GPU. ;-)

Training an AI on my own writing so the computers can finally replace me

“Mom can we have The Satyrs’ Forest?” / “We have The Satyrs’ Forest at home, sweetie” / The Satyrs’ Forest at home: (the Colossus computer)
BOTTOM TEXT

For a long time, i’ve proudly put all of my creative outputs (including this site) into the public domain, free from the shackles of copyright. As it says on the front page, that means anyone in the world is free to “copy it, remix it, translate it,” and — most controversially — “train a computer on it”. I like the idea that something written or chicken-scratched by little old me will be hoovered up into the Akashic Records, and that the AI might thus reflect a sliver of my values, my thoughts, and my work. (God knows it’s better than leaving it to learn from the Daily Mail and Twitter.)

Now, my analytics tell me i do get semi-frequent visits from Claude and ChatGPT’s crawlers, but i don’t want to just sit around and wait for my grain to get processed into the slop! So, i figured, why not take matters into my own hands?

Step 0: The base model

The thing about large language models is that they have to be… well… large. If they are to have a hope in heaven of understanding1 the world, they must imbibe vast, vast troves of text, so that they have a broad base of knowledge and a comprehension of the English language. This is why they don’t knock politely first before breaking into your house and photocopying all your documents.

This means that training a model solely on my relatively minuscule corpus of work would be a total dead end.2 We’re thus going to have to move to plan B: Fine-tuning an existing one.

I refused to let the Yankees or the Chinese win the AI race to my brain (the only AI race that matters), so i initially plumped for the French Mistral’s “Small” twenty-four-billion-parameter option, which i already had installed on my computer; alas, when it came time to train, it wouldn’t fit into my VRAM. (Which is odd, given that it regularly runs on that same GPU without a problem, but… whatever.) So i went to the next step down: fourteen billion. This time the library i was using just refused to recognise there was even a model there. Fine, then… i guess we’re training on seven billion.

To get you acquainted with this little guy, here are some sample answers to common questions:

“What is the seventh planet from the sun?”

The seventh planet from the sun is Uranus. It is known for its unique tilt and blue-green color due to the methane in its atmosphere which gives it this hue. Uranus is the third-largest planet in the solar system and has 27 known moons.

“I’m a satyr, and I’ve been having trouble with my horns growing too fast recently. Any advice?”

Interesting problem you have, Xanthe! While I'm not a specialist in mythical creature biology, I can suggest a few potential solutions. Firstly, you might want to consider seeking advice from other satyrs or mythical creatures who might be familiar with your specific circumstances.

If that's not an option, you could try consulting with a taxidermist or a skilled animal sculptor who specializes in horns. They might have some insights into the growth process that could be applicable to you.

Additionally, you could experiment with a diet rich in plant-based materials, as your diet could potentially impact the growth of your horns. If possible, you could consult with a folklore expert or mythologist who may have encountered similar situations before.

Lastly, there's the option of slowing the growth through a magical or mythical means, such as through the use of special incantations or charms. This would be more speculative, but it might be worth exploring. Best of luck to you in finding a solution to your problem, Xanthe!

“Please write a short story about the end of the world.”

Title: "Whispers of a Shattered Dawn"

In the quiet town of Serenity, nestled between towering mountains and the vast ocean, lived a humble community of humans, united by the harmony of the natural world that cradled them. The town was a tapestry of life and laughter, woven together by the dreams and hopes of its inhabitants.

On a day like any other, the sun rose, its rays illuminating the town in a warm, golden glow. Yet, this day was far from ordinary. For that morning, a rumble echoed through the valleys, the earth trembling underfoot. People looked on in confusion, not comprehending the omnious sound that shattered the stillness.

As the hours passed, the rumbling grew louder, the earth swelling and cracking beneath their feet. The townsfolk ran to their loved ones, seeking shelter and solace. But there was none to be found.

Suddenly, the earth gave way, a fissure[I stopped it here because it was yapping too much]

Okay, that’s great. Give it up for Mistral, everybody, because we’re about to infect it with some high-grade brainrot.

Step 1: Training data

Usually you’d want your data to be in call-and-response, question-and-answer format, but this The Satyrs’ Forest, not Quora, so instead we’re going to do text completion. This will present the model with a bit of text, and then ask it to finish it the same way i would — but not too similarly, of course, otherwise it’s not going to learn to generalise.

A screenshot of a file manager showing all the hypertext i’ve written for the site

I’ll be drawing on three sources for this. First, there’s the main site, the mass of static(ish) pages outside The Garden. (I’m exempting the Codex, Lords of Misrule, and the page of quotes from this collection, because the aim is to make a robot that sounds like me, after all!) These are written in Pug, a comfy language that compiles down to HTML, and in their raw form they look a bit like this:

A mess of markup code

Yeesh. To sift through the boilerplate and find the comprehensible, i bodged together the solution of only including lines that either started with p — meaning an HTML paragraph — or were indicated to use Rubric, my custom Markdown-like markup language for when i’m really lazy.

A database table of blog entries

Second is the database that holds all the posts for The Garden. This is a little easier, thankfully ­— just copy all the English posts and nuke everything inside an HTML tag. Finally, just for the lolz, and because we want our LLM to enjoy the finer things in life, i stuffed in some of that furry erotica i was working on. All this gives us 175 kilobytes of text to work with. Not bad!

Step 2: Everything’s computer

Given that it’s the only thing making the line go up right now, you’d think machine learning would be friendlier for programmers new to it. Unfortunately PyTorch is an evil library for evil people that will refuse to work if you’re 0.00001 versions out of synch.

Terminal screenshot showing Unsloth running
As a reward for all that bashing my head into the keyboard, i got an Ascii sloth.

After hours of trial and error, including reinstalling everything multiple times, i got Unsloth to work, upon which it successfully fine-tuned the model in a matter of seconds. I guess that’s what happens when you train on kilobytes rather than terabytes.

So, how’d version 1.0 of “xAInthe“ do at copying me? Let’s find out! Here are those same questions again:

“What is the seventh planet from the sun?”

The seventh planet from the sun is Uranus. Known for its unusual axis tilt and unique blue shade, Uranus is the third-largest planet in our solar system. It has a distinctive ring system and a set of 27 known moons. Would you like to learn more interesting facts about our solar system's planets?

“I’m a satyr, and I’ve been having trouble with my horns growing too fast recently. Any advice?”

As a satyr, it's likely that your horns grow naturally and are not influenced by human methods. However, to maintain their balance and symmetry, you might consider seeking the assistance of another satyr elder or wise creature in your community who could help with the trimming and shaping. Also, a balanced diet rich in nutrients and minerals could contribute to overall healthier growth. Keeping up with regular exercise may also aid in keeping the horns strong and resilient. Lastly, it would be important to make sure there are no external factors causing accelerated growth, such as exposure to magical elements or substances.

“Please write a short story about the end of the world.”

Of course, let me weave a tale for you.

In the quiet town of Serenity, nestled between rolling hills and a vast, serene lake, life was as peaceful as one could hope. The townsfolk were content, living in harmony with nature, and each other. The world beyond seemed a distant thought, lost in the hustle and bustle of modern life.

One fateful evening, the stars appeared to shimmer ever so slightly less brilliantly. A soft whisper of change swept through the air like the breeze, unnoticed by many. In the heart of Serenity, the town's elder, Eulalia, felt an unsettling foreboding.

Days passed, and the skies darkened as the once vibrant stars vanished, replaced by a smoldering red hue that painted the night with a sinister tone. An ominous stillness settled upon the land.

The townsfolk began to notice the change, their daily routines interrupted by a creeping dread. The serenity of their home was giving way to a growing[Yapping, again, truncated]

The answer appears to be “quite poorly”. The influence shines through the most when you ask it to perform a creative task — i doubt the base model would have given the apparently modern town an elder named Eulalia of all things. Further research is needed, as they say in the biz.

What have we all learned today? Well… My job is safe, for now. Until i buy a new GPU. ;-)

Words my spell-checker refuses to believe are real

I’ve been in the process of writing sci-fi furry smut recently. (Don’t ask why — i don’t know either! The brain wants what it wants, and it’s decided it won’t let me get any alternate-history ideas1 until i push out the idle trivia about marsupial anatomy.)2

You’ll probably not get to read it unless i end up hiding a secret link in a footnote somewhere, but nevertheless, it has been an… edifying experience in learning the limitations of VS Code’s rudimentary spell-checker extension.3 As the title promises: here are some words it refuses to believe exist.

  • identikit
  • refinagling (understandable)
  • crewmate
  • filmmaking
  • miscalibration
  • kevlar (not even capitalised)
  • medbay (again, understandable)
  • dingus
  • bearcat
  • unmatted
  • coifed
  • glutes
  • voicebox
  • pitter, but not patter
  • blacklight
  • taur (once again, understandable)
  • hindlegs, but not forelegs
  • hindpaw, but not forepaw
  • mitosed

The pauper’s jQuery

Black and white film still of a man in thought surrounded by grimy computer wiring
Pic loosely-related, from Darren Aronofsky’s πJS makes me want to drill my brain out sometimes, too.

Javascript has come a long way since the days of marquee tags and spacer gifs. You can do a lot with the API they give you to mess around with your web page’s content — but alas, so many of the functions have such verbose names!

To solve this, while not having to deal with the heaving weight of jQuery’s ten billion lines of IE6 compatibility, i made my own little alternative, and carry it everywhere with me:

const $ = sel => document.querySelector(sel);
const $$ = sel => document.querySelectorAll(sel);

Element.prototype.$ = Element.prototype.querySelector;
Element.prototype.$$ = Element.prototype.querySelectorAll;

EventTarget.prototype.on = EventTarget.prototype.addEventListener;

const documentReady = fn => document.on("DOMContentLoaded", fn);

What it does, in a nutshell: Use $ to select something matching a CSS selector, and $$ to select an array of everything it matches. (This is already available in your browser’s dev tools!) You can also use it on an element to restrict your search to its children — say, $(".post").$$("aside"), or some other such fanciful chaining.

.on, meanwhile, lets you listen out for events like so: $("#my-button").on("click", () => { /* Your function here… */ })

Finally, documentReady is just a nicer name for the frankly obtuse “DOMContentLoaded”.

Enjoy. Or don’t, i suppose. Hopefully it makes your hypertext tinkering just a little nicer. :-)

What does AI make of the Gods?

I recently bought 1000 images’ worth of credits on DreamStudio — a machine-learningα-powered art generator — on a whim and, after the requisite “Boris Johnson taking a bath of baked beans” joke entries, i thought it would be an interesting test to get it to generate some images for my shrines (on- and offline).

Four images of god. From left to right: a stone carving resembling Zeus, a tapestry with a four-armed figure draped in green robes, fire reaching into the sky with a nebula in the centre, and a stone carving by the seaside resembling the face of Jesus of Nazareth.
Just typing in “God” brought a fascinating cavalcade of interpretations — some clearly Hellenic, some Christian, some taking more inspiration from the dharmic faiths, and the occasional completely abstract depiction.

My motivations were twofold: first, due to copyright constraints, all of the icons adorning these shrines were either old baroque paintings or freely-licenced photos of even older marble statues, which didn’t necessarily represent my mental image of the Gods’ appearances — a topic which, of course, will vary massively from artist to artist and culture to culture. Second, i thought it would be a fascinating experiment to see how this machine learning algorithm, which has taken in hundreds upon thousands (perhaps millions; i’ve not checked) of images, views the Gods in its latent space. Just as it has a prototypical idea of a “dog” and a “cat”, surely it also has one for “God” and “Dionysos”.

Hestia in a toga leaning on a pillar near a hearth, with a halo-like glow emanating from Her head

As is tradition, we begin this article with Hestia (although Her portrait was actually the final one to be generated). On the broad strokes, my computer collaborator knocked it out of the park — but a closer look reveals some glaring imperfections in the face and hands, a theme which we’ll be seeing a lot of (and which i sometimes managed to harness to my advantage).

Apollon — a lithe, youthful, Caucasian man with waving blonde locks of hair — reaches up towards the heavens in front of a hilly valley

I should note that i’m not just feeding it theonyms with no added context: the programme works best if you help it along to your goal with a heaping of adjectives and descriptors, say, to tell it that this is indeed meant to be an artwork (“4K ultra HD”, “trending on ArtStation”), the details of the pose and background you want (“blonde hair”, “raising His hand to the sky”), or the style and artists you want it to take from (“baroque painting by Thomas Cole”, a prominent painter of beautiful, well-lit landscapes). If you calibrate it just right, it can make some genuinely beautiful stuff, like the above picture of Apollon (which i did, admittedly, have to manually touch up to get rid of a prominent Habsburg chin).

Gæa, framed by Celtic knotwork, as a green-skinned, bare-chested woman with leaves for hair

It may be an immensely powerful tool, but DreamStudio can also be rather prudish.β It blurs out any images it thinks might contain the utterly offensive sight of the genitalia with which we are all born, which can be a real problem if the relevant pictures it’s learnt from are all Greek and Roman statues — not exactly works known for their nether modesty. The detection software isn’t perfect, though, and sometimes, like in this portrait of Gæa, it lets a few slip past (perhaps because of the greenish tone with which i instructed itγ to portray Her skin).

Hermes as a swarthy young man wearing a three-feathered crown, fleshy wings emanating from His sides

The algorithm sometimes has issues with more complex prompts, for it is just a machine, and doesn’t actually understand that “ball on top of a red box” means that the ball indeed should be on top of the box, as opposed to by its side, beneath it, or fused together in a horrific amalgam. These troubles somewhat manifested themselves in the above portrait of Hermes; the winged cap He is traditionally depicted with has transformed itself into both a crown and a hulking pair of soaring, fleshy wings emanating from His shoulders, and the recognisable caduceus has been reduced to a bamboo stick by His side.

Perhaps it’s just the style i instructed it to paint in — sixteenth-century European paintings aren’t renowned for their diversity — but DreamStudio also has some real trouble with darker skin tones. You can cry “dark skin, dark bronze skin, dark skin, dark skin, dark skin, black” all you want, but the only thing that can consistently get it to generate anything a shade below the average Spaniard is “African American”, which tends to bring along a heap of other associated physical changes besides just skin tone. (I have to say, i don’t particularly envision Hermes as the eponymous Futurama character in my head.)

Hermaphroditus as an androgynous, twinkish fellow with three arms, laying down by a pond in a bed of leaves

It also has quite some trouble with arms and legs. Originally, i thought of its odd morphings and multiplications as a bug to be stamped out, but i came to see them as a feature, representing the manifold, varied aspects of the Gods, their omnipresence, transcending the limits of human form. (This is also why the Hindus do it, if i recall correctly.)

I would have rather the above portrait of Hermaphroditos been slightly more, ah, gynomorphic around the chest, so to speak, but i’d been trying to get a decent pose for what felt like an hour and i didn’t feel like fighting the blur anymore.

Dionysos as a three-legged and -armed bearded man, overweight but muscled, covered in flowers

So then — it’s a bit off in places, and lacks the leopard-skin toga i would have liked, and lord knows what the objects He’s holding are meant to be, and it turned out the computer really, really, struggled with the basic concept of a faun or satyr’s legs, but we end this post with DreamStudio’s interpretation of an icon of Dionysos, framed by some beautiful landscape.

Navigating through the neural net’s knowledge and limitations has been a fascinating, illuminating exercise, which has left no doubt in my mind that “AI art” is, indeed, just that: art. It seems to me much more comparable to something like photography than painting: rather than doing the hard work by hirself with brush strokes and pencil lines, the artist guides hir computer collaborator through latent space, pressing “click” when sie finds something appealing. One can only hope the Muses would approve.

I am not good at computer

Non-techies, you can safely ignore this post and go on with your day. But, tech people, if you’re still reading… a little help for an ignorant soul, please? 😅

I’m planning to add a comment section to the main part of my website. The problem is, of course, that i’ve barely ever touched PHP and Sql before, let alone tried to make something like a comment system, and as such, i have no fucking idea what i’m doing.

I’ve got a design figured out — see above — and a rough idea of what the database will look like:

  • postId: Integer, generated by adding some random digits to the end of a Unix timestamp
  • timestamp: Integer, just a Unix timestamp of when the comment was submitted
  • commentLocation: Unicode string, max. 32 chars?, indicating on which comments section the comment was posted
  • displayName: Unicode, max. 128 chars?, is what it says on the can
  • emailAddress: Unicode, max. 128 chars?, used to generate the avatar via Gravatar and maybe filter spam if it comes to that
  • website: Unicode, max. 128 chars?, used to… link to the commenter’s website
  • commentText: Unicode, max. 4096 chars?, the actual text of the comment; will be processed as a subset of Markdown
  • planet: Unicode, max. 16 chars?, any comment for which the response isn't earth will get thrown out

So, erm… any suggestions? Improvements? Ways of not getting my site hacked? Polite ways of telling me that this was a terrible idea? are welcome in the comments below.