Jon Kågström

I always wanted to create a chatbot, but I never got around to actually doing it since I wouldn’t want to write anything where I have to explicitly give it knowledge or manually feed it with language rules. Every time I’ve started to sketch – I kind of got stuck being unable to understand how to overcome this.

Lately I had another urge (after watching Prometheus) so once again I started by sketching on paper trying to understand how to attack the problem. Then, after a few evenings I decided just to start coding. Remembering the proverb “programming is understanding”.

First I started by looking on Wikipedia, using classifiers to give contexts etc, but it became clear that it would be beneficial to start by looking on the syntactic structures of language.

So I instead of jumping into the semantics (which is what I really like to play with) I decided to take a step back and do some syntactic stuff first. I reformed my plan to what I’m working after today:

1) Parse a dictionary with tagged words and samples
2) Extract grammar rules from the samples (memory based model)
3) Create a random jabber bot using the grammar rules (syntactic correct nonsense)
4) Use the grammar rules to resolve sentence ambiguity
5) Make synonyms, antonyms and derived meanings part of the system (allows for better context understanding – you can also play the association game with the bot after this) (<=== this is what i'm working on at the moment)
6) Make it possible to add context stimuli to the jabber bot. For example talking to it about a tree, will make it jabber about forests etc.
This will hopefully start to resemble about the hybrid in Battlestar Galactica. Even though it’s talking a lot of nonsense the “nonsense” is within a semantic context.

“The five lights of the apocalypse rising struggling towards the light, the sins revealed only to those who enter the temple only to the chosen one.”

7) Add simple associative memories. (From here and on I am not sure how I will implement yet)
8) Train it on conversational transcripts (question-answer patterns etc)
9) Feed it with knowledge (from wikipedia)

I will now shortly summarize what I’ve done so far:

1) Building a dictionary from Wiktionary

So I downloaded Wiktionary (wiki – dictionary) instead and used the SAX parser to extract information about words and their classes. The cool thing about Wiktionary is that it’s constantly expanding, it has tons of words, their definitions, examples, synonyms, antonyms, attributes, phrases etc. The bad thing is that wiki pages are not built for machines but for humans. So parsing pages require some extra time. Anyhow I spent a day building the parser and tagging the common word classes (verbs, nouns, pronouns etc) extracting examples and such, so it wasn’t too bad.

2) Extracting grammar rules from Wiktionary

As I mentioned earlier, most words comes with one example or more – my idea was to generate language rules from this. With a language rule i mean a syntactic correct series of word classes. For example, “the table is red”, the rules would be “article->noun->verb->adjective” then I would store this as a valid rule. Of course, since words have many meanings, classes and different origins (etymologies) it’s not that simple.

Another possible rule extraction from “the table is red” would be “adverb->verb->verb->noun” since ‘the’ can be either an article or adverb, ‘table’ a noun or verb (to table something) and ‘red’ the describing color or the actual red color.

This was a bit worrying at first, but after running the rule extraction on large amounts of data it turned out that I could extract a lot of unambiguous rule chains by using unambiguous words. In addition I disambiguated a couple of common words. For example “he” can be the personal pronoun or “The name of the fifth letter of many Semitic alphabets (Phoenician, Aramaic, Hebrew, Syriac, Arabic and others).” – removing the noun meaning exposes “he” phrases for rule extraction. By doing so on the articles, personal pronouns and the three most common verbs (be, have and do) I think it is good enough (some rules will be errornous but for this experiment I consider it some “noise”). I don’t strive for perfect.

Here is an example of an extracted rule:

Artificial [_start]=>Article [Definite]=>Noun [Countable]=>Verb [SingularFirstPerson, Past, Auxiliary, SingularThirdPerson]=>Article [Indefinite]=>Noun [Countable]=>Punctuation [Period]=>

3) Create a jabber bot

This was easy – just randomize words following random rules. Rules can also be merged using those that maximize the overlap of each other. Here is the first words it said:
“this has then gather next me” After fixing some bugs and and improving the rule extraction this is some of the things it randomly says now (including periods):

“i have an hybrid orbital instead of the test bench…. they are cold turkey rascally ladings are each who semi-annually skidded…”

“she meant me an efficacious oligomenorrhea. a solidarity does chemical castration hersirs. they wear me. she nitrogenized ironically.”

“he was monotonous during his lyophilization.”

“an twinning pigeon hole chihuahua”

“inconsistency secondary hypothermia was an sunni gino-sho near the orecchiette , who was an alpha privative.”

“the mustaches have an swallowtail, interjoin megawatts, wildly.”

Just like anyone who is learning English the bot mixes up a/an and also some of the rule merges breaks the English. However, I’m not aiming to make a grammatically correct bot. I want to create something that can make sense in a semantic manner – therefore I’m not spending time on the nitty gritty details.

4) Sentence disambiguation

When talking to the bot it needs to know what you mean, to know this sentences needs to be resolved to their meaning. As a first step in this I use the existing rule library to score the possible ambiguities. The scoring function matches each combination between the possible rules, where the amount of overlap is measured.

For example “huge ants are wondering about life” is resolved with the highest score to:
huge Adjective []
ants Noun [Plural, Countable]
are Verb [PluralSecondPerson, PluralFirstPerson, PluralThirdPerson, Auxiliary, SingularSecondPerson]
wondering Verb [PresentParticiples]
about Preposition []
life Noun [Countable]

All the attributes above (within the []) are automatically extracted from Wiktionary.

Anyways, this method is not 100% and I think regular POS taggers (parse trees) might do better, but again, this is good enough for now.

Hopefully I will take this project further. I would like to “upload” the bot (java) to a server so it can be tested by everyone while developing, but maintaining it will take time too.. Will see where to go from here.

Jon :: Jun.30.2012 :: Uncategorized :: No Comments »

Not much happening on my personal blog, but here is a short update.

For the last three years I’ve been working at king.com with game development, however now I’m switching to work at mojang instead, probably with minecraft. I’m really happy and psyched about this and looking forward to work with Markus, Jakob, Aron again as well as getting to know the rest of the crew. In case you missed minecraft:

As for my other projects here is some stuff I’ve been working on lately:

playchilla.com

I’ve had most fun fiddling with some game development lately, nothing extraordinary I just do whatever comes to mind. You can check it out on playchilla.com where I blog a bit of what I’m doing, right now I’m putting together a tower defense game.

uclassify.com

Also I’ve been doing some work on uClassify, I have nearly finished implementing the keywords API that I’ve tested with this visualization. Whats left is to add it to the URL API and document it. Will try to do this in January 2012.

Jon :: Dec.27.2011 :: Misc :: No Comments »

Editorial notes: Yet another guest insight by Dr. Put that only took me 5 months to insert into this blog. Sorry about that, yours truly Dr. Pull.

It’s only now – finally being able to relax, stretching out besides the main pool of my hazienda “Maria Dolores” in southern Tijuana- that I am able to put all the events of the past weeks into perspective.

Changes

Let me start by telling you a little story about two daoistic monks that were on their way to the highest situated monastery named Maoshan. Shortly before they reached the wooden gate, a flock of birds shat on their robes repeatedly changing the colour of the latter from orange to a yellowy white. While the disciple struggled, the master remained calm and smiled. And finally he answered the disciples asking look with the words: “It’s good to change”. Rest assured there might be some moral to that story. As per usual.

In Dantes kitchen, where the pullability crew had been lost for almost two weeks, everything changed with the arrival of PM “The Nighttrain” Kågström. You know what they say about nighttrains: There is no stopping them and they always reach their goal. Oh, rest assured indeed! Looking back we accomplished more in those three days of joint effort than in the two weeks before. By only giving up a few unnecessary practices like “writing the blog”, “sleeping” and “eating”. Easy, so easy.

And the face of the kitchen started to change….

PM “The Nighttrain” Kågström is now an honorary board member of PI Ltd.

Distractions

Any great men doing any great deed face constant distraction from different corners of temptation. And it takes a lot of resting assured and per usual-ing to resist them. Dr. Puts Achilles heel was Swedish women. One in particular. And as Hakan from the Ironshop says: “Building with a travelling mind is like riding a blind horse”. Luckily Hakans speciality is Newton and not horses. Or women. Or building for that matter.
Dr. Pulls Achilles heel was Dr. Put being a useless, push-uppy, mindwandering snail. So, in order to focus Dr. Puts efforts, he treated PI Ltd’s number 1 to Swedish cinematic greatness: films like “Brüno” and “Year one” made ALL other options look like a Japanese Lotus graden. Even putting up tiles.

Tiles

“It’s down that road. I know it!”-“How?”-“Well, I have a feeling..” And suddenly you find yourself wandering the outskirts of Stockholm in blazing sunlight in search of the mighty “Kakelspecialist”. It sounds like an old man of great wisdom when all it really is is a shop full of toilets and tiles. But look at the way we were treated: Wearing our stinky builders outfit, that already proved advantageous in getting us some free seats in the T-bana, once again did the job. We were treated differently than the well-dressed Stockholmians. We were treated by the staff as one of them. Which was fun…until they started to use their crazy “builders lingo”. It was then that Dr. Pull showed his true greatness by saying: “We want to put up some tiles. Give us everything we need for that.” And it was also then that the store clerk realised he could sell us anything including useless, ugly-coloured buckets, sponges and tools that are still assessed by PI Ltd’s intelligence department for their function. For all the people that are using this blog as a guide to build their own kitchen (which is an estimated 36% of the world population), one more resting assured: Do NOT carry 3m2 of tiles. EVER. Take a taxi and the driver might just be a retired kitchen builder himself (no kidding!).

Mosebacke

“Where it all starts is where it all ends” is true in so many aspects of this Swedish adventure of Dr Put. With a tear in my eye and a rare selection of endangered animals on the grill, I shall close my platinum Mac Book Pro and go for a game of Polo against the Syrian Royalty. But I could not possibly leave you all without some final advice: It’s really in the surroundings! (as crap golf-players say)

I don’t think Pullability Industries Ltd. could have done such a great job without the help of so many people: Specialists from Canada were flown in to inspect the work at different stages and used shark-metaphors in their advice.

An ACTUAL true kitchen specialist sawed the crucial holes. In the sands of Thailand, Dr.Pull-Brother and his lovely girl (the two of them have a REAL kitchen. And a Jacuzzi! Why am I Dr. Pull’s friend again and not his brothers?) supported us. There was even people making us feel not-so-bad about being hungover (thanks Johanna!) but bad about our slackline skills (thanks Roger!) A martial artist who almost put half of PI Ltd. out of service. (the same half of PI Ltd. who got his ass kicked in armwrestling by a girl just minutes before). A hippie from Urkult who can’t keep track of his dates. A combustion engineer who had a hybrid taxi explained to him. The Nightrain and the wattenpass. And last but least: The Chief Designer! Without her, we would have been like taxidrivers with no onboard navigation. Like trains in the wrong tunnel. Like Joon and Phil. Thanks, Mia! Our hearts are beating like jungle drums.

Curtain call

May life write another chapter in our books of life. In Paris. I promise.

Jon :: Dec.28.2009 :: Rebuild :: No Comments »

If you have lived in Sweden your whole life, you might know about this. You might even rest assured. All the others shall tremble and shiver along with me when hearing about this.

Swedish craftsmanship

For lack of having proper colonies, the Swedish king (we have all seen his picture wearing a skirt!) has ordered all ceilings in the capital to be built like African bush-huts. Woven -bloody- grass is underneath the gips! One might ask oneself: how does a Swiss guy know what´s behind Swedish gipsputz? Could it be possible that he barely even looked at the ceiling and a huge chunk of the roof fell down smashing the floor next to him and -once the dust and debris had settled- revealing tribal hut construction underneath. Rest assured that yes, it is possible (see pictures underneath).

When the area next to the hole was gently touched to be shown to the apartment owner, the hole (enjoying being in the spotlight) decided to spontaneously expand. Let´s hope that all people interested in ever buying this apartment have stopped reading this text a few lines back.

But as much as they have holes in this country, they also have the best solutions to fix them: ”Husfix™”. A poisonous, irritating, lung-up-fucking material that can fix EVERYTHING. Even Newtons ”Law of Gravitation”. Rest assured that there is always a physicist at hand at your local ”iron shop”.

Recovery

One thing we enforce at PI Ltd. (Pullability Industrial Limited) is proper recovery of the workers on a site. There is several ways to ensure the human resource is at full charge:

• Coca Cola (a small, cute, fjun spin-off company of PI Ltd.)
• Recovery (available in different flavors)
• Trocadero
• a wide range of widely unhealthy food

Impress with progress

This 3-picture-collage shall show the progress we are making:

Jon :: Aug.03.2009 :: Rebuild :: 2 Comments »

After a good night of sleep, dreaming about destroying kitchen parts, we successfully woke up and reinitiated the kitchen rebuilding. As we are still in “Project Mayhem” most of our work will be of destructional nature.

A glimpse into the past

This blog series does not only serve as a practical guide for (for amaturs up to experts) of how to rebuild your own kitchen – it’s also educational. Therefore I will try to inject some interesting facts as we go along. The house itself, nicely situated on Hökens gata close to Mosebacke, was built in 1858, this was the year after the big fire on Södermalm that burnt down all houses (even the theatre). I think this also was the same decade our olden king Gustav Vasa liberated Sweden from some occuping forces. Considering how long ago this was (just look at the silly beard), it was to no surprise that the floor looked really bad as it was probably installed in the same epoch (cheap floors during wars). Then all the spilling of food on it for many decades has certainly left it in a very bad shape. So as we are doing this properly, rest assured that we are building a new floor.
This is Gustav Vasa:

And here is the olden floor:

New floor made of cutted planks

Luckily we found some planks laying around. Unluckely the planks didn’t fit on the floor so we had to cut them first.
Cutting planks is really easy if you have the right tools & knowledge (gathered during project Hawkeye). In short, cutting a plank can be done in two ways, either by hand or by a cutting-machine. We got a cutting-machine to do the work for us as we didn’t want to get blisters in our hands.
Look and learn:

Risk assesments

Rebuilding a kitchen can be dangerous if you don’t assess the risks properly as you go along. This is not only a part of “Project Hawk Eye” (see the mission plan above) but also an iterative process that you should take seriously throughout the operation. This is our system: Everytime we encountered a risk one the encountee loudly calls out for a risk assessment: “Risk assessment”. Then the other one carefully approaches the risk and we evaluate it together. One example of this is electricity. After taking safety precautions along our risk assessment we proceed. Here we can see how Philip is carefully detaching a on-off button, note the plastic handle on the screwdriver, the gloves and the face mask (spark protection) :

*don’t try this yourself unless you really know what you are doing (feel free to ask us in the comments if you are unsure)

Day 2 Summary

We did a really good work today, cutting a floor and neutralizing risks, we have learn alot about electricity and wood cutting (easy peasy lemon squeezy). We are not sure what to do next, either we start to reroute the waterpipes or perhaps paint something (would be nice to start on “project Eden” as it’s probably even more fun).

Rest assured that this journey of insight and knowledge will continue.

Jon :: Jul.29.2009 :: Rebuild :: No Comments »

Today I will initiate a kitchen rebuilding project, starting by removing the current kitchen and then putting a new one in (also a kitchen). Todo this I’ve called upon an old friend from Switzerlands – namely – Philip.

Planning is A and O (as we say in Sweden)

Since I have some prior building experience I know that planning is the most important part, understanding exactly what to remove and what to put in. Some people even use some advanced instruments (such as yardsticks) to measure that new cobbars acctually fit, sometimes this is done in cm but some people use inches (I guess inches are used for greater distances, such as hospitals). But I don’t think those overly-complicated tools are necessary for us, as we accually have building experience, also, I don’t think “the naked chef” uses a ruler every time he slices a bread.

So what I did was a plan with some blueprints. Since I don’t have PowerPoint (which I understand that most architects use to draw rooms and even houses) I did it by hand on a board. This is me revealing the plan to Philip as he just arrived:

And here is a close up of the actual plan (feel free to reuse it, I’ve put the plan in the Public Domain)

We divided it into three parts, project Hawk Eye, project Mayhem and project Eden, even though this may sound complicated it is very intiutivly once you get a hang of it. Rest assured that you will understand it when this blog series is over.

Project Mayhem – day 1

Since I stretched a muscle in my back, Philip had to do all the heavy lifting, we can clearly see that here:

And here:

Rest assured that the oven was carried to the garbage room and that Phils back is ok.
Here are some more pictures to inspire you to rebuild your home:

This is what we managed to produce during day one:

As you can see we cunningly left the sink with all the pipes going around, and I admit that the reason is that we don’t know what to do with all the water that is going to spray when we hammer them. But rest assured that we will deal with them in a future blog post once we have read the water pipe instructions on wikipedia. My guess is that we have a many towels (at least 10 big ones (not the small hand towels)) and a few buckets. So one of us manages the water collection into the buckets while the other reroute the pipes. But that is just a guess so don’t try that. Look on wikipedia first.

Team picture of two true handymen:

Jon :: Jul.28.2009 :: Rebuild :: 2 Comments »

Det har sannerligen varit ett fantastiskt år med många välspelade tangentnedslag. Nu är det dock dags för semester ända till på Måndag och var är semestern bättre än i Zurich?

Mikrofonen i telefonen har gått sönder men jag lyssnar gärna om någon vill prata. Kanske DTMF toner går att använda? Det har jag inte provat än. Om det fungerar så betyder ett pip JA och två NEJ.

Att resa utan mobil innebär naturligvis rigorösa förberedelser – precis som på 90-talet. Lyckligtvis har Philip gett mig tydliga anvisningar att infinna mig på tågstationen och där uppsöka en telefonkiosk. Hans noggranna beskrivning, efter det jag placerat mig på lagom avstånd till en mynttelefon, fortsätter

”call this number: 044 265 09 24. The number has to be entered exactly and in the order portrayed.”

Haha, han är briljant.

Jon :: Nov.19.2008 :: Svenska :: No Comments »

Tillsammans med Roger och Emil har vi precis fått klar en beta av en gratis webtjänst där man kan skapa egna textklassificerare, uclassify.com! Kan blir riktigt häftigt

Här en site som användare tjänsten www.ofaust.com. Den klassificerar texter mot en massa författare. Då kan den nyfikna se vem man är mest lik i stilen!

En av mina gamla poster (funkar bara på engelska) matchar närmast Nietzsche, men bara till 16%.

admin :: Sep.30.2008 :: Svenska :: 1 Comment »

Lördag, filmkamera, mia, jon, koltrast, edith, “erik”, regn, balkong.

Jon :: Jul.19.2008 :: Svenska :: No Comments »

Eftersom denna veckans ordranson (ganska liten som norrlänning) börjar ta slut så får bilderna tala (jag villl kunna kommunicera lite i helgen också).

 Bilen.

 Middagen.

 Ölen.

 Jon.

 Emil.

 Grabbarna.

 Hänget.

admin :: Sep.13.2007 :: Svenska :: No Comments »