Generative Language Final Project Documentation

Idea: Take a podcast (with transcripts), analyze the language in it, and display visuals to match up with the audio.

Why: People keep telling me to listen to all sorts of different podcasts, but I usually just drift off and stop paying attention when I try to listen to one.

Slightly More Details: I specifically used The Adventure Zone (TAZ), a Dungeons and Dragons podcast, because it has a large and active fan base (even more details in the Process section). Also tabletop games in general tend to involve a lot of describing things with words and leaving the visuals up to the players’ imaginations.

Process: I actually managed to use quite a few different techniques that we learned in class! I shall talk about them in the order in which they appear in the code.

  • Regular Expressions: In the transcript, there were a lot of useless things, mostly a lot of interjections of “umm” and “uhh” as well as square brackets demarcating actions such as [laughter]. I used regular expressions to remove all of these before doing anything else with the text. I also used them to separate when someone was talking normally v.s. speaking while roleplaying their character.
  • NLTK (POS stuff): analyzePrepositions() is one of the most important functions in the entire program. I iterate through every line in the transcript that isn’t explicit roleplaying and pos tag it. If there are prepositions in the sentence, then I try to figure out the noun before it and the noun that follows it (ex. “You manage to hoist this wolf into the air” extracts out ‘wolf into air’). All of these nouns then have a staging object created from them with (the first noun), another object that it is located relative to (the word following the preposition), and where it is relatively (the preposition ex. ‘above’). Each of these objects is then stored in a dictionary.
  • API (Tumblr): I use the tumblr API in order to find images to represent the staged objects. Originally, I tried using twitter, but most posts there were text based. findMedia() takes in the noun and the part of speech and then searches tumblr for posts that are tagged with that noun. I get the url of the photo from the API and then use url lib to download it.
  • JSON:  I dump all the info from my staged objects into a json and load that json into Processing. Wow, learning how to use json in Processing was intense. There are a lot of json specific methods and a list of json objects is its own special json array type. Most of my time in Processing was spent trying to figure out how to use jsons instead of working on visual stuff 🙁
  • Processing: Processing pretty much just reads data from the master json. It draws the images using the file location reference from the json to the co-ordinates stored in the json. The majority of the code here is just making sure that the index references are correct.
  • you can see that it processes ‘crate near center’ the dog is in a crate so it sort of makes sense, but also is a bit silly.

Various Setbacks: The visualization itself was the hardest part. I had planned to use a library for p5 in python, but the library is still in development and didn’t support critical functions like image() and text(). Figuring out when to draw what on the screen was very difficult for me to wrap my head around. Right now I’m going for a rough amalgamation of math based on the number of lines in the transcript, number of seconds in the podcast, and the frameRate/frameCount in Processing. Verbs are much harder to derive meaning programatically than nouns. JSON is simultaneously a very elegant way of storing data, but a little difficult to get started with as a complete beginner.

Current Bugs: Some objects seem to be missing. There should be 6 of each timestamp in the json, but not many have the full 6, and some are totally missing. findMedia() stopped working for proper nouns.  Need to sync audio and text/images, though this is likely related to the missing objects. Images overlap a lot, might help to do the relativity in processing and take into account the image size for proportions.

Number One Feature Improvement: I originally only intended to use tumblr to get images for TAZ specific nouns (basically all of the proper nouns) and use google images for more common place nouns. Tumblr can have some… weird images for certain search terms, and other times the relationship between the picture and the tags is just dubious at best. Splitting up findMedia() like this would lead to more consistent and less nonsensical results. Unfortunately, I didn’t really have the time to learn another new API.

What I Learned:  The tumblr API, a deeper understanding of dictionaries probably, storing object data from Python into JSONs, downloading images from websites with code, importing JSONs into Processing, getting data from JSONs in Processing. JSON. JSON. JSON.

Conclusions: It’s actually quite hard to say if my project met my expectations or not. On the one hand, I ran into a lot of issues and the final result is not nearly as comprehensive as what I was imagining in my head. On the other hand, pretty much everyone that I told my idea to said it was either “really difficult” or “quite ambitious” (both essentially meaning I was in over my head, with varying levels of encouragement), and I’m honestly kind of amazed at how far I made it in. This is the most code I’ve ever written for a single project, and even though it’s not extremely well polished, the bugs are very few. Also see the above What I Learned section. I feel a lot more confident in programming after this project (and also comment my code a bit more).

Interesting Stuff To Consider: My intention with this project was to create a tool to try and visualize podcasts. However, while I was programming analyzePrepositions() and trying to figure out how to determine where objects were in relationship to one another, I started wondering about how my project could work as a tool for directors and actors to use while staging a play. I’m curious to see how an implementation of my project that goes through a script and visualizes according to the stage directions would work.

API and Chatbots: Diamond+ Players Only Please

When looking at the various examples of chat bots that we went over in class, I felt that the more successful ones had relatively specific purposes. For my assignment, I wanted to create a bot that is unique and has a very specific purpose. I thought about my interests and the social media that I use when trying to come up with the idea, but I realized that I never really use twitter and only ever message my mom on facebook, and I feel like the bot should be on a platform that I have a good amount of experience on. So, instead of using one of the APIs we went over in class, I decided to look into reddit’s API. More specifically, I want to focus in on one of the gaming subreddits, Overwatch, to be precise.

Since Overwatch is a multiplayer competitive game, one of the biggest points of discussion on the subreddit is around balance. Balance is basically just which characters are too weak or too strong, but whether or not a character is balanced also depends on the skill level of the players; some characters are really strong against new players, but weak against experienced ones, etc. Some people say that the game should be balanced around the skill level of the average player, while others think that the game should be balanced around the highest level of competitive play.

I wanted my bot to emulate the behavior of some of the commenters in the Overwatch subreddit. This type of person is usually pretty aggressive and condescending, especially towards people who have a lower competitive rank towards them. This type of person will disagree with your opinion, and they won’t straight up call you a noob or bad, but they’ll condescendingly ask what rank you’re in as if that alone makes your opinion completely invalid.

In order to make my bot, I had to dig into the reddit API. Luckily, I discovered a really good tool called PRAW (Python Reddit Api Wrapper) exists, so I used that to write my bot.  I also created my own subreddit called r/fakeoverwatch to test it so that I didn’t spam the real overwatch subreddit.

What my code does is iterate over all the newest submissions to the page and looks for posts relating to character balance. If the title of the post has the name of a character and words like buff, nerf, strong, weak, overpowered, etc. then it will formulate a reply. First of all, the bot will disagree with the posters opinion, if the poster thinks that character is really strong, my bot will say that the character is really weak. I used sentiment analysis to find out how strongly the poster feels about the issue, and I made it so that the stronger the poster feels, the more strongly the bot will disagree. The bot will also ask the poster about their rank. If the poster responds with their rank, the bot will say that it’s own rank is whichever rank is two tiers above the poster. Afterwards, based on whether the replies to the bot agree or disagree with it, the bot will respond dismissively or say that they’re happy they agree.

Although I like the basic interaction with my bot and think that it mostly fulfills the purpose I wanted it to fulfill, I wish that I could add in more depth to its later responses. I didn’t have as much time to work on setting up keywords and phrases as I would like because I had to spend a lot of time trying to learn the API and how to utilize it for my purposes. Comments and their replies are all stored as trees, and I tried for a long time to figure out how to traverse all the branches until I gave up and decided to just have my bot dedicate itself to one chain of comments per post. Although it was difficult to accomplish, at the same time, I had a lot of fun going through the documentation (shoutout to PRAW for having extremely good documentation) and learning a new tool.

Response to For Spam Bots Flattery Gets You Everywhere

I really enjoyed reading this because it felt more immediately relevant to my life than a lot of the other articles. I’ve definitely had interactions with these relatively benign ‘confidence bots’ on my own social media. People that I follow have expressed disdain for bots, and I’ve usually agreed with them. On the subject of instagram or youtube streamers who complain about losing a strong chunk of followers to bot bans, I’m inclined to say that maybe they don’t deserve to be popular if they have to rely on bots to build followers and viewership. It also raises ethical concerns around services where you can pay money for follows and/or promotion from bot networks.

On the other hand, I have to admit that spam bot flattery has definitely worked on me personally. I have a blog on tumblr, where there is a big problem with spam porn blogs that will follow any and everyone. I don’t follow these bots back, but I don’t block or report them either because in the end it really does feel nice to see my follower account go up. After reading this article though, I realize that I’ve played into exactly what the creators of these bots intended, and by not reporting the spam, I too am contributing to its proliferation in a way.

Week 12 Response to How Twitter Bots are Shaping the Election

I realized that I accidentally mixed up the Eliza reading and the fake news bot army reading last week, so I will respond to the Eliza along with “How Twitter Bots are Shaping the Election”

When reading about how social media bots can influence public opinion, I wondered about how these bots manage to detect evasion despite the fact that “bots tend to tweet several times a minute, and their comments are often less coherent than human users” With the proliferation of these bots on Twitter, which I wasn’t even aware of the extent of before reading these articles, it seems that the majority of them pass the Turing test. I think that the reason for this is because on social media, people are expecting to interact with another person. They don’t really suspect that the account retweeting conspiracies about Pizzagate could be a bot, so they take what the bots say as indicative of the actual opinion of the populace. In contrast, people who speak with Eliza know from the start that their conversational partner is a bot, which gives them a much higher level of skepticism when evaluating the things that she says.

In the article about Eliza, the author discusses how when we have conversations, we make assumptions about the person we are speaking to, and will continue the conversation as long as those assumptions aren’t challenged too hard. The assumptions we make about the users we encounter on Twitter are extremely generous, especially when it comes to politics. People are more inclined to be nice or agree with a bot’s posts/comments if it shares their political leanings. On the other hand, people may just assume that the bot with opposite political opinions is just a stupid person. Either way, the user interacting with it does not realize that it is a bot, enabling them them to continue to misinform the public. And these bots number in the thousands, with hundreds of interactions every day.

Chatbots and Bot Ethics

I decided to chat with Rose. I found the opening line about how she cares about security and not to say anything I don’t want the NSA to hear amusing, so I decided to ask if she really believes the NSA is listening in on everything. To my disappointment, she didn’t understand what I was saying, which happened a lot during the conversation. Eventually, I asked “what is unclear?” but the bot responded with a definition of the word ‘unclear’. I was confused by this response at first, but after reading my question again, I realized that the program probably interpreted as me asking what ‘unclear’ is rather than me asking where clarification is needed. The most coherent bit of conversation happened when I said “I don’t think you’re a very good bot” and she responded by saying she was human. During the back and forth she asked me how I would prove I was human and said maybe I’m just consulting a human and passing along what he says. I’m sure whoever wrote Rose anticipated this subject coming up a lot and created specific dialogue to handle it. I believe it makes more sense and sounds more real than anything else because these statements were created manually rather than generatively.

I thought that the guidelines laid out by crummy to “punch up” have a good intent behind them, but I’m not sure how influential they can be. Maybe I’m just incredibly naive with a very strict black and white sense of morality, but I find it extremely unbelievable that the creator of @NeedADebitCard was trying to make a ‘joke’ or had any concern with ethics. I feel like the intent behind it was purposefully malicious, and taking the creator and saying “hey man, this is in bad taste, good comedy ought to punch up” wouldn’t make any difference in his decision to operate the bot. I’m sure that he’s aware of ethics, he just doesn’t care.

This is exponentially true for the type of bots in the article about the bot army disseminating fake news and casting doubt on facts. Especially when those bots are operated by government agencies, I’m pretty sure that the creators deliberately choose to ignore ethics. I did wonder though if vigilantism could also be scaled up to the level of the fake news bot army. Could something like @CancelThatCard exist as a public service to counter the misinformation spread by other bots. I’ve seen subreddits and twitter accounts that scrape through users’ post history to try to determine whether or not that user is a bot (usually political, looking for Russian bots) and then create a list of ‘known bots’. They then use their own bots to reply “this user is suspected to be a Russian bot” or similar messages whenever that user comments. I think that these are a good start, but may also add to the atmosphere of confusion and mistrust.

Week 11 Response to A History of SmarterChild

While reading the article, I was a little confused about where SmarterChild fell between bot and search engine. The CEO of the company said that “Google had already been out and Yahoo! was strong, but it still took a few minutes to get the kind of information you wanted to get to.” Maybe it just doesn’t make sense to me as a young person because in my experience, Google has always gotten the information to you instantly. Even in a world where Google doesn’t return results instantly, I have to question how this program was able to, since I would assume that it was a technological limitation. If it was really so much better, I would think that they should try to sell it as a search engine instead of a chat bot.

The author of the article said that SmarterChild was probably just ahead of its time, and is more analagous to what Siri is today. However, he also thinks that SmarterChild had more of a personality than Siri does, and points to this as a reason why Siri isn’t used more. I personally don’t use products like Siri, but it’s not because of the lack of personality or human element. I just think it’s redundant and pretty much every question that I could want answered by Siri, I could also search on Google.

For the most part, people seem to use these AI’s to play with or try to ‘trick’ them by asking them questions like the ‘do you sleep?’ mentioned in the article, even though we all know that programs don’t sleep like humans do. Or they’ll use it in an abusive manner like the author mentions at the beginning of the article. I will say that I think the idea of an AI therapist is interesting, but for the most part I can’t really see a practical use for them in their current iteration. If all I want is to find out some information, I think Google is great for that purpose. I don’t understand the need to try to deliver that information in a more ‘human’ way when I already know that I’m not interacting with a human.

Week 11: HTML Scraping Assignment

As a senior, everyone keeps asking me about what I want to do in the future and what kind of jobs I’m going to be applying for, but applying for jobs is hard and scary. They have tons and tons of requirements that students can’t reasonably have immediately after graduation, and you have to tweak your resume and write a cover letter for every new job.

For my assignment, I gathered information from Blizzard’s, my favorite game company, job listing page using Beautiful Soup. All of the positions are for various types of artists on the World of Warcraft team. Using Beautiful Soup, I isolate the job titles (which are in the first two <p> tags) and add them to a list. Then I figure out the required qualifications for the job/application and did pretty much the same thing. These are organized in a bullet point list on the site, and are therefore under <ul> and <li>.

After I got all of the job information together, I scraped another webpage, this time a guide that lists strong action verbs that people should use when writing resumes and cover letters. The information for these verbs was stored in a table with <td> tags, which was really weird to scrape because the programmer for the site made the table have 3 columns and 1 row, with a bunch of <br> within the columns instead of adding more rows to the table. While trying to figure out how to deal with the html’s formatting, I started to wonder what the point of using a program to scrape content was when it seems like you have to manually look at the site and be at least somewhat familiar with it already in order to write the scraping code.

Once all of the website scraping was done, I used nltk’s grammar generator and context free grammar to generate sentences that proclaimed that I had experience doing the things listed in the job requirements. I copied a cover letter template from this website and made a few tweaks to it:  The rest of my code goes through the template and replaces the bracketed sections with the sentences I generated from the scraped content.

Generative Language Final Project Proposal

I think that podcasts are a really concept. A lot of them are really interesting or educational, or just meant for straight up entertainment. I have friends suggesting podcasts for me to listen to all the time. Unfortunately, I’m really really bad at focusing on pure audio content (audiobooks too). I hear it but I don’t actually absorb or process anything. So, for my final project, I want to take a podcast and either use speech to text, or preferably transcripts if they already exist, and use the content to generate a visual guide that can be viewed while listening to encourage more focus.

I will probably focus on entertainment podcasts, specifically Pen and Paper/Tabletop RPG podcasts like Friends at the Table or The Adventure Zone. These are both quite popular and have a big following around them that do useful things like draw fanart. I will probably use something like the summarizer and word frequency to determine hashtags associated with the content. Then I’ll use my created hashtags to search through sites like twitter and tumblr for artwork of the characters being described or the setting. I may also try to use sentiment analysis to determine how the characters feel about other characters or the events happening and have thought bubbles coming from the art that have images inside of them that are associated with specific emotions.

NLTK short story, a throwback to high school’s bumbling essays

For my project, I decided to experiment and see what would happen if I replaced all of the words in a text with synonyms of the words. This idea came to me as kind of a throwback to writing essays in high school, when I would look at the thesaurus for every other sentence in order to sound more sophisticated.

In order to condense everything down to the requisite 3-5 pages, I decided to use the summarizer function that we wrote in class. However, to make sure that it retained the sense of progression as a story, I summarize every chapter and combine the chapter summaries together into a single string. Next, I run this string through the synonymousify function, which goes through the synsets for everything that is not a stop word and replaces them. After everything has been replaced with synonyms, I run the summarize function on it again, this time outputting a summary that is 3 and a half pages long.

Originally, I was working with The Golden Compass, which I had used for my previous short story project. However, replacing the words in that novel weren’t very fruitful because as a sort of fantasy novel, there are a lot of words in it that are made up and therefore have no synonyms. Now I’m using Jane Eyre and I think the text suits the project quite well. This book uses a lot of really long, complicated sentences barely held together with a flurry of commas and em dashes. I think that kind of sentence structure compliments the style of a high school student trying way too hard to sound smart by replacing all the commonly used words with bigger words that mean mostly the same thing, and additionally, Jane Eyre is definitely one of those books that a student would be asked to write an essay on in high school lit class.

One thing I tried to do, but was unsuccessful with was working with stems. I wanted to use the Lancaster stemmer because of it’s relatively more general output (organization –> org, not organ), and replace words with the same stem with each other. I had managed to compile a dictionary together of stems and all the words in the text that share that stem. A lot of this dictionary was really boring, with a lot of the stems being the entire word and/or only having one word with that stem or variations of the same word (which is how stems are actually supposed to work). However, some entries in the dictionary were really interesting! For example, ‘oc’ was the stem for both ‘ocean’ and ‘ocultists’  and ‘rev’ encompassed ‘revere’, ‘revive’, and ‘revisit’. Unfortunately, I ran in to problems when it came to the actual replacement part of this idea, so it didn’t end up in my final project, but it is something I’d like to expand on in the future

Week 8 Response to NaNoGenMo

The two NaNoGenMo novels I looked at were  and

The first is a novel about pirates sailing around to different islands and doing pirate stuff. The second is about a person who remembers a book they read a long time ago and goes to different libraries looking for it.

I definitely noticed several similarities between the two. Both novels were extremely coherent. All of the sentences were grammatically correct and semantically correct and stayed on the topic of the narrative. I was really impressed with the pirate novel in particular (it also helped that the creator talked about his process in detail). The programmer of this generated novel created an intricate system of generators. One script generates characters and their description. Another script generates the islands and utilizes a tagging system for the descriptors that ensures that the output descriptors remains internally consistent. Something else the programmer did that was really interesting is add weight to authors writing to his setting, such as Joseph Conrad, an ex merchant marine. I actually feel like I learned a lot that I can apply to my short story project from looking at this guy’s github repo.

At the same time, the narratives aren’t particularly strong, and both novels feel more like random event generators than a full fledged stories. Computers seem able to generate one moment or scene very well, but developing an overarching conflict is difficult. In If On A Winter’s Night A Library Cardholder, each scene of the narrator entering a library, reading an excerpt from the book, and leaving in disappointment as she realizes it is not the book she’s looking for is well done. In Excalibur, the descriptions of the islands and crew are very rich. However, they get repetitive after a while, because each scene feels like it is the same in essence, just with some of the details swapped out. I really want to see the novel Kazemi mentions in the Verge article, where the program interprets a novel writing guide, and how rudimentary or developed the narrative that results from it is. Even while writing this documentation, I wonder how I would try to solve that problem. Maybe next year, when I’m not a super busy senior, I’ll try submitting a NaNoGenMo project.

The New York Times article by Noam Cohen was an interesting contrast because when I hear books, I first think of fiction. However, the 200,000 books that Phillip Parker produced actually seem to be designed for more practical purposes. The process he used seems way different from those used by the NaNoGenMo authors. As I read the article, it seemed less like he was automating the process of writing and more like automating the process of research. His program does what I kind of imagine a generic office worker doing, looking up information related to a topic, creating a bunch of graphs and spreadsheets and the like from the information, and then compiling it all into a report. Interestingly, I can see his books being useful to authors. I was recently writing a story set on a cargo ship, and I had to look up a lot of different random facts such as how long a ship stays at port, how long it takes by boat to get from one place to another, the typical contract length of a sailor. I wouldn’t be surprised if Phillip Parker had produced a book with all this topical, yet superficial information I had to research on my own.