We thought it would be good to collect some of your stories about how the cipher challenge has affected you, how you tackled some of the challenges, and what it has meant to take part. If you have hints or tips about how to run a good cipher team, or a code-breaking club, or any ideas for lessons based on the competition then get in touch at [email protected] and if you want to find out more about how it works or what to do then download our competition guide and codebreaking handbook linked above.
The Book Brothers
We started doing the challenge aged 12 and 10 as part of a small team and still enjoy it today as just the two of us, 5 years later. We have even guest starred as librarians of the Cipher Challenge website, creating a list of cipher and spy based books. Whether you are a mathematician, problem solver or just enjoy a good quiz this is for you.
You can participate in this challenge as an individual or as part of a team, when you start you are guided through some practise problems to get you thinking in the right way and before you know it you are hooked.
There is nothing more rewarding than finally getting from cipher text to plain text after many hours of head scratching and clue finding. The Cipher Challenge has brought us many emotions as our hours of confusion and anger turned to pure joy as we kept trying, found leads and ultimately solved the ciphers. There is something mysterious about this challenge that means that when you start you just can’t seem to stop, it is always there at the back of your mind and it won’t stop pestering you until you finally solve it.
Another thing that we love about The Cipher Challenge is how beginner friendly it is. Before we started The Cipher Challenge we knew nothing about ciphers or programming, everything that we know today is from the challenges we have completed. This means that you can take your time when doing the challenges and use online tools if you don’t know how to program and this makes it appealing to everyone regardless of your skill level.
Harry is the Cipher Master, your guide through the challenges and giver of clues as time ticks away. You are never quite sure if you love or hate him as his clues always seem to come just too late or are just too cryptic, but he always helps out in the end. There is a great community in the forums where people try to help each other out (just not until they have solved it themselves and bagged the points).
The final amazing thing about The Cipher Challenge is the story. Every year the amazing team at Southampton University don’t only make a challenging set of ciphers but they also write an amazing historical story to go along with it. These stories are part of what makes the challenge so enjoyable, you learn and live the lives of the people, start to think like them, predict what happens next. This is one of the key things that makes us come back year after year, to learn about another historical event, whether it is war time Russia, Ancient Rome or a mission to the moon.
Thank you to harry and the team, Bring on The Lighthouse Conspiracy.
HS Crypto Crackers
We were introduced to codebreaking in January 2013 and decided to try it with our Maths club. Having no codebreaking experience ourselves, along with our group of 5 enthusiastic S1 pupils, we taught ourselves about cryptography from resources we found online and muddled (rather successfully actually) through. We all became totally captivated by codebreaking and in August 2013, we decided to start a dedicated club. CryptoCrackers was born!
When we’re not competing, we spend the rest of our time practising our skills. From pulling ‘Crypto Crackers’ at our Christmas party (yes, we really did replace the jokes in our Christmas Crackers with coded messages that we made for each other!) to learning about computer coding so we can hopefully improve our performance in next year’s National Cipher Challenge, we never miss a chance.
Becoming a cipher master
We started the Cipher Club at my school in 2018 and our Thursday sessions have genuinely been the highlight of my weeks since. It’s given me a great opportunity to get to know some of the students that I didn’t teach at the time and it’s given us something to test our brains a bit. The challenges start off easy and there’s guides on the website for anyone who is worried about not having done this before. I took one of those puzzle magazines from WHSmith on holiday with me in the summer before I started and that set me up perfectly. The stories written by the team are great and often historically accurate, which a few times led us to say afterwards “Oh how did we not realise that!” when we figured out the keyword. The best part of it though has been spending time with enthusiastic students who want to be there.
We have so many inside jokes now, if another teacher comes in to see what we’re doing, they often leave confused as to why we’re laughing and how we’re making any sense of the letters that are on the board. Even when the challenge finishes in January, we still meet every Thursday to do puzzles or play games. Sporcle.com and Geoguessr.com are some of our favourites and then we had a phase of playing Among Us too.
In 2020 when the lockdown hit, we carried on through Zoom. This is a testament to how good the Cipher Challenge is and how much of an impact it’s had on us. The team created a special challenge for everyone so we had something to do and didn’t have to wait until September for it to start again. Once again, this was something we looked forward to each week. We’ve never won the competition as we’ve never been able to complete the last part, it’s always so difficult with some weird two part cipher that we’d never even think about. But that’s never stopped us enjoying it and we always give it a go.
I’ve even been inspired to write my own puzzles for the students, including riddles, a cryptex and a treasure hunt.
That was a lot of fun to create and even more fun to watch them solve it, it took them 3 weeks. We had to say goodbye to three of our students leaving with Year 11 this year and this was my way of thanking them for sticking with it and providing us with so much entertainment. The icing on the cake however was when these three students returned the favour and created a cipher and treasure hunt for me! It included puzzles, riddles and the traditional Cipher Club snack of Jaffa Cakes! At the end of it they had made a three page, hand written cipher for me. It took me 4 hours to decipher and it documented the last three years of the club, all the ups and downs, inside jokes, successes and failures that we’d all been through. It’s the best gift I’ve ever received.
The work that the guys at Southampton University have done to run this challenge is incredible and it’s providing so much more than just a puzzle to solve on a Thursday night. I would encourage any staff to find the time to get a small team together on a Thursday night after school and have a go at the challenge. It takes no preparation, just do a quick Google search on what Substitution and Caesar Shift ciphers are and you’re good to go. I hope you have as much fun as I have had and will continue to have in the future.
Planes, trains and automobiles (Phones, coaches and ferries)
Claire was one of the winners of the 2017 National Cipher Challenge, and as usual we asked her to write an account of how she broke the final challenge. Her story was, to say the least, unusual!
There is something particularly satisfying about participating in the Cipher Challenge. Perhaps this is because of that moment of relief when the incomprehensible gibberish of the ciphertext finally resolves into a readable plaintext; or perhaps it is because every year there is the chance to make your cracking techniques more refined, your programs more efficient, your solving times a little faster than before. While the ability to write a Python program which can automatically solve Vigenere ciphers may not (in general) be the most useful skill, the Cipher Challenge encouraged me to keep on looking for ways to experiment and improve on my own previous work, trying even the most unlikely solutions to fix increasingly minor problems. Although I am, sadly, now too old to officially compete, the Cipher Challenge remains an excellent way to take a break from regular academic work; and the fact that I, among others, continue to be persistently hanging around on the forums and leaderboards long after leaving school is a clear testament to what an enjoyable and stimulating experience the Cipher Challenge is.
Some of my original methods for facing the final challenge in 2017 were a bit strange! This was mainly a result of being on a school trip when it came out, so having a limited amount of time to do it and an even more limited amount of time with a computer available to do it. This meant that I had to rely a lot less on programs than I would have liked – although I don’t think that this actually worked out too badly for me in the end.
Initially, without access to a computer I looked at the cipher on my phone (ignoring part A, on the logic it would be easy with my programs). However, when I finally had access to a laptop, about three hours after the challenge was released, I put the text into my ioc program (Ioc.py) and discovered that it wasn’t a digraphic substitution cipher at all. At this stage, I fairly easily managed to do 8A, only to discover that it didn’t really tell me anything helpful.
I ran a lot of other tests on the cipher then, including frequency analysis (letters, digraphs, trigrams, words) (Frequencycount2+.py) and ioc (every 2nd-20th letter, digraphs, both of these after simple transpositions), but didn’t find anything helpful – apart from confirmation that both ‘QN’ and ‘LA’ were common overall. I briefly considered the autokey cipher, as a cipher I had come across when researching the beaufort cipher for an earlier challenge; but after trying ‘agricola’ as the first word and receiving nonsense, I gave up on that approach. I then had a few hours when I couldn’t do the cipher. Later that evening, I had more time to look at it, so ran yet more fruitless data analysis, taking photos of all the results so that I could look at them even without the computer (unfortunately I have since deleted these photos, so cannot remember all the random things I tried). One of the things I did try was finding all the one letter words and looking at the letters before and after them, and their position in the text (Letterfind.py). I seemed to find nothing conclusive with this, although I did find some patterns between the three letters – for a bit of text ‘…A B C…’, sometimes A+B-C seemed to give me the letter ‘i’, sometimes A+C-B seemed to give me sense – but these patterns were inconsistent and didn’t give me anything truly helpful. Nothing I tried early the next morning seemed to work either.
Somehow I then managed to ignore the challenge until about half past three, when I was on a coach.
I don’t really know what made me try it – possibly I’d never truly let go of the idea of autokey – but I tried shifting the word ‘the’ one place relative to itself and adding the letters, e.g. t+?, h+t, e+h, e+?. With some surprise I discovered this gave me ?AL? – the reverse of the ‘LA’ which I had thought was ‘an’. Trying the same thing with ‘and’, I discovered that it gave me ?NQ?, the other common pair I had noted. At this stage, I knew I was getting somewhere – it appeared that ciphertext = plaintext + key, with the key being the plaintext shifted by one letter, as in an autokey cipher with a one-letter keyword, and so plaintext = ciphertext – key (key = previous plaintext character) – just, sadly, I’d made this discovery half an hour into a fourteen hour coach journey, for at least eleven hours of which I would not have access to my laptop.
Obviously, as my digraphs were reversed, I needed to reverse the ciphertext – unsurprising as 8A had been reversed, which I had thought might be a hint. Wanting to be able to decode from one end, I began with the fourth word, ‘LAX’ (forgetting, I think, that this was actually going to be the end of the text) and successfully decoded this as ‘eht’. However, when I tried to progress onto the previous word, ‘BXHWEI’, I got nonsense (‘?mlwae’). Assuming I’d made a mistake, I tried reversing the end to give ‘PKFCO ENPKMRG YAL GT OOBQHVJ’, and turned ‘YAL’ into ‘the’, but again, trying to do the other words I only got nonsense. Although ‘LAX’ looked like it could be somehow coded within itself, as it need the key ‘hte’, the same letters as the plaintext, I knew that the words could not be independently coded as ‘XAL’ and ‘YAL’ both translated as ‘the’. This was when I thought that reversing the whole text was probably too easy, and thought that maybe individual words were reversed – which also explained giving us the word spacing, which otherwise seemed far too nice (my original thought had been, ‘Oh no, it’s so hard that even with spaces we won’t get it’). This new idea was successful, and allowed me to decrypt the end of the text, ‘of the ancient world’. However, I encountered yet another problem with the start; I managed to decode from ‘the’ back to ‘??spite the’, but was then getting ‘iaspite’. I guessed that the correct word was either ‘despite’ or ‘inspite’; ‘despite’ seeming better, I chose to just accept the anomaly with the e and continued decoding, discovering that the phrase was in fact ‘yearning for respite the’. After continuing to decode manually, marking every letter which seemed to decode oddly, I noticed that it was consistently every twelfth letter of the text, which was effectively encoded with the letter ‘a’ regardless of what the key letter should have been. I was lucky to make the discovery then, as it meant that when I eventually wrote a program I could account for this.
At this stage, I still had about ten hours until I would be able to use my laptop, so I proceeded to spend the night trying to decode it manually – which, on a coach, working by the light of my phone in the middle of the night, was not at all fun. At the end of the night, I had only decoded about a third of the text, but at this point I was able to spend an hour and a half on the ferry using my laptop. In this time I wrote a (very glitchy) program (2017 8Bx) to solve the cipher, and wasted more time trying to solve the glitches; when I got off the ferry I realised that I could use the glitchy program to solve the cipher, in conjunction with a vigenere program (Vigenere.py) I already had.
Therefore, when I got home I ran the program – the first major problem was that it could not run to completion, but I simply told it to print the plaintext after every iteration, and I had already manually decoded the final phrase; and for some inexplicable reason, the even characters had been moved six down (I had made the false assumption that ord(“a”) and ord(“A”) were 26 apart), which was solved using my vigenere program. I had only been able to tell what had happened to alternate letters due to knowing how the plaintext should start: in this way having already decoded the start was helpful, although I did not need anywhere near as much correct text as I had.
I then only had to find the secret message. I was suspicious of the use of the word ‘yearning’, as it seemed a bit of an odd choice, so I assumed that the ‘y’ was the first letter. I wasted more time than I should have done looking for evenly spaced letters, before it hit me that punctuation must have been given for a reason. After comparing my deciphered, punctuationless text with the punctuated original, I found the message, and realised that I’d finished, far quicker than I’d expected. At no stage did I think of the 12×12 hill cipher; I was treating the cipher as a variant of the autokey, as this seemed to make sense with the other ‘imperial’ ciphers.
City Learning Centres seek to inspire teachers and students to use ICT in innovative and creative ways to enhance the curriculum. We also try to challenge students of different abilities to achieve more that they believe is possible. I use the National Cipher Challenge with Nottingham students to address these aims.
The National Curriculum in Primary Schools often reduces the teaching of maths to SATS preparation, mental maths and a few investigations. Able mathematicians are often isolated in their schools and have few opportunities to work with similar students. Each Summer I run a Summer School for students in years 6-9 to spend a week problem solving cryptographic puzzles. I aim to create an atmosphere where being good at maths is great! It is competitive, unashamedly geeky, very quirky and, I hope, fun. Students form bonds with similarly talented students which they can develop through a Saturday morning G&T maths classes at Nottingham University for the rest of the Secondary School life.
Enrichment activities for maths are numerous but the Cipher Challenge is particularly good because:
It lasts for a term with a natural momentum which builds to a climax, encouraging students to develop their interest.
It has a additional literary or historical focus
Requires students to research
Requires and develops tenacity
Develops bonds between students and allows some to shine and lead
Extends outside the classroom to other schools and students
I run a crypto club at the South Nottingham CLC 2 nights a week. A small but very dedicated group of students mixed ability, mixed age meet to solve problems, participate in the challenge and eat biscuits. They support each other. They are often teased by other students for volunteering to do maths after school but I support them with strategies to counteract that. They also come in during the holidays to solve harder messages later in the challenge. We are in touch with similar students and groups at other Nottingham Schools and support each other.
I have run teachers’ sessions and also advertise the challenge with maths PGCE students. I have run sessions at crypto clubs in other schools. Often teachers find that the messages in the challenge become too hard for them and can feel uncomfortable with that. Students do need help to solve later messages so, either by email or phone, I support teachers with possible strategies to try. Students become really frustrated if they can’t progress and often only require a simple but appropriate hint. I have tried to encourage students to use the forum but as the Challenge forum is frequented by very able students who are often dismissive of less able students they find it daunting. The clues there are often too cryptic for our students. The experiences of many of our students are restricted to celebrity and popular culture. Last year there were references to Shakespeare and Marlowe which were totally unfamiliar to them. This is one of the Challenges strengths and reasons why I continue to use it.
I also run days when schools can bring a class of students to the centre, learn about cryptography and participate in a mini challenge. We also write messages and challenge each other with them. Using maths in a different context is a new experience for students and helps them realise that for some adults, maths is a career life choice. Maths is something they do because they have to and is generally outside their career choices so this broadens their horizons. It is determination rather than ability that helps the students succeed.
Our students really enjoyed the Cipher Presentation at Bletchley Park. Thank-you for organising it and providing the opportunity for them to experience something so stimulating. After Simon Singh’s talk Kim uttered the immortal words, “he’s really inspired me to do more maths”. I couldn’t ask for a better reaction than that.
At crypto club students often tell me that “I never thought I could do that”, following success. It isn’t solving messages that matters but the journey they take to arrive at the solution. After they succeed they often marvel at the wrong turns they took and how they decided what to do next.
They don’t always cope well with the pressure and tension that builds in the room. Sometimes we have to revisit a problem 10 times before they make progress, but the eventual sense of achievement is worth the struggle.
Several students are inclined to give up if they can see a method (eg. longhand looking at each morsecode signal and transcribing it alphabetically) that is tedious and avoid starting, hoping that someone else will do the work. Eventually they persevere and get the job done – a useful skill for later life.
Teamworking is often required to break final messages in the challenge, trusting each other to do a job accurately and well is a valuable experience.
Some of my favourite moments occur when a year 7 student makes a breakthrough that has defied a year 10 student. This is a humbling experience for the older students and a fantastic morale booster for the younger. Mutual respect for each other is generated.
Quiet, able students in busy, inner-city secondary schools often survive by becoming invisible. The cipher challenge gives them a unique mathematical opportunity to gather together, share ideas and think.
I entered the National Cipher Challenge in 2013, 2014, 2015 and 2016, with notable successes of second place with my team in 2015, and first place in 2016 as a solo entrant! My result catapulted me into a mini competition held by Netcraft where I started to learn more about the world of cyber security, particularly the dark side of phishing and how fraudsters deploy and receive credentials by using phishing kits. This captured my interest and I swiftly interviewed and accepted an internship with them which I undertook during the summers before and during university.
My route into Netcraft is not a typical one; I first got into programming through creating mods for the popular video game Minecraft, then programming algorithms to assist me in competing in the Cipher Challenge, creating a fully-fledged UI desktop program complete with its own API (https://cipher.tools/). I did not study Computer Science at A-Level nor at University, I instead studied Pure Mathematics at Warwick with only 3 CS based modules, so the vast majority of my computing knowledge is self-taught or has been learned on the job. I believe that you can only learn so much from theory prior to entering the workplace, and a lot of the learning comes from hands-on experience working on production codebases. Software development is all about planning, design, and creativity – programming is just the tool to create the automatic procedures and execute the plan. Therefore, no matter your background, you just need to have the interest and commitment to keep at it – qualities that the Cipher Challenge instils in you. It helps you to develop a wide variety of skills to independently absorb information, translate it into algorithms and then write code, and of course the sneaky gremlins at Southampton still make you do manual work and use your brainpower on the final challenges, testing the limits of everything you know.
This problem solving is why I signed on to be full time with Netcraft as of June 2020, where my day-to-day work involves working on a system which takes down around 300 incidents of malicious content per hour, with high levels of automation. There is a constant need to evolve our systems to be able to handle new types of threat, as the fraudsters change their behaviour.
I have come a long way since participating in the National Cipher Challenge and it turns out that in most cyber security career paths you will not be dealing with Caesar ciphers on a daily basis! What the Cipher Challenge does is give you the transferable skills and problem-solving abilities needed to pursue a successful career in cyber security. Doing something that not everybody does will immediately make you stand out to potential future employers. So, I implore you, give the challenge a go or keep going if this is your second or third year. These things take time and hard work but the satisfaction of completing a difficult challenge is well worth it!
James Lloyd, Trinity College Cambridge
What I enjoyed the most about the cipher challenge was the thrill of discovering patterns, but then building tools to speed up or automate the pattern discovery process. The sudden excitement of an “aha” moment and the satisfaction of watching a well-oiled machine do its thing. And now, lucky for me, I get to build machines to automate pattern discovery for a living. After the cipher challenge I studied mathematics at Trinity College, Cambridge, and my experience from the challenge certainly helped by giving me a head start in programming and optimisation algorithms. During my study I specialised towards statistics, i.e. one of the ways to formally study pattern discovery, continuing to an M.Phil in statistics. Along the way I spent some of my holidays working at a hedge fund (searching for patterns in financial data) but after an enjoyable internship I joined the Boston Consulting Group (BCG) as a strategy consultant. Part of my reasoning for joining BCG was a desire not to be typecast as a “mathmo”, but I soon realised that I rather enjoyed being a mathematical specialist, so I contacted my M.Phil advisor, Richard Samworth, and asked what PhDs might be a good fit for me. After speaking with some of the people Richard recommended to me, it was ultimately David Spiegelhalter who tipped me off to the existence of the Machine Learning Group in the Engineering Department at Cambridge. I discovered I could study my favourite flavours of statistics at the time (Bayesian and nonparametric) but also be working on artificial intelligence, which sounded rather exciting. Studying for a PhD with Zoubin Ghahramani in the Machine Learning Group was a great decision. I loved going deep into areas of mathematics and getting to the forefront of human knowledge, I got involved in some fun side projects (I helped write a musical using statistics and machine learning) and I got to meet plenty of smart and interesting people, including my wife (also a Trinitarian). And fundamentally, my work was all about automating the process of discovering and understanding patterns. Since my PhD I cofounded an employee survey company, called Qlearsite, where amongst other things I applied the latest advances in natural language processing to the task of discovering what topics a company’s employees are talking about. It is very satisfying watching someone genuinely understand what is important to their employees in a matter of minutes whilst knowing just how much computation had to take place in order to make this possible. Recently I moved to Silicon Valley so I could live permanently with my wife and I now work at Abacus.AI, a machine learning platform startup, where I’m leading their efforts to build anomaly detection and natural language processing systems. My journey of automating pattern discovery continues.
Some of the leading agents who worked on the 2016 case wrote up how they tackled it. If you are wondering what it takes to become an elite cryptanalyst at BOSS read how these agents tackled a message encrypted with a brand new code. A lot of the challenges you will face as a BOSS agent will not be anything like as difficult, but as for the later ones …
Notes from our debrief of Agent Cribbage, one of the team who broke open our 2018 case. Cribbage explains his thinking as he tackled another brand new code. His explanation includes a nice brief introduction to the idea of the index of coincidence, one of the more sophisticated tools in the codebreaker’s toolkit.
A guide to solving Challenge 9B from the 2019 training module, written by Challenge archivist, Madness. He maintains an archive of all of the old challenge texts over on Github and if you search for them you will find them. He is also writing a long and interesting book about how to break the ciphers you see here and many more besides. Every time I look it has grown. If you ask nicely he might let you see it too.