Very Deep Not Boring Beginner Projects
Very Deep Not Boring Beginner Projects
I get a lot of requests for projects a beginner can accomplish, but usually I give out simple little things that someone can do to build their initial skills. That's fine for people who can't quite code yet, but what about people who just realized they can code but have no ideas? This post is a list of projects that you can do, won't be boring, and have a ton of depth to them despite being approachable.
This is a Living Document
If you're looking for a new project then check back to this page periodically to see if I've added a new idea. Also, if you have an idea for a style of project that fits then email me at help@learncodethehardway.com with your idea. I'm looking for descriptions of something you actually did and what you actually learned. What I don't want is projects you heard are good from someone else and didn't actually try.
What is "Depth"?
All of these projects ideas are gateways to subjects which have deep educational opportunities. This doesn't mean that these projects are difficult, it just means that if you can get into one of these then there's a whole world of things you can learn after that.
Let's take the Log Analyzer idea as an example. On the surface this doesn't seem very complex. You write a program that reads each line of a log file and parse it right? Yes, that is really all your first log analyzer needs to do. However, log analysis has become a massive subject all on its own because of a fairly recent movement into utilizing logs to analyze entire industries. Log analysis now has elements of statistics, data mining, distributed computing, and database design. That means if you keep trying to push your understanding of log analysis you could have projects and topics to learn for a decade or more.
At the same time, just because log analysis can be complicated doesn't mean that you can't make a simple one to get started. That's the point of these projects. They are simple introductions to complex topics in programming.
General Advice at the End
NOTE: After all of the project descriptions I have a bunch of commentary on things that may thwart or discourage you. It's important you read that, but I moved it to the end so that the projects are more prominent. After you browse the projects and find one you like, scroll down to after them and read what I have to say about Gifted Perfectionists, Famous Programmers, and Publishing Your Work.
Project 0: Reimplement Anything
Finally! The damn projects! The first project is kind of a meta idea that if you're looking for something to create, then copying something else is a fantastic age old practice since the beginning of humans. In the art world we call this a "master study" and the practice is to take a painting by a much better painter and attempt a faithful replica of it. Doing this teaches you so much about how to paint that it's one of the top components of a classical art education.
I recommend you pick small tools that either you need but can't find a version for your OS, or that you find interesting and want to study. I'll give you an example:
I switched to Windows almost exclusively years ago and I've found alternatives to almost every command line tool except for less
. I've been tempted to write my own because I desperately need one, so maybe you can do it. less
is a pager, which means it takes input from other programs and displays it in pages that fit on the terminal. You'll usually hit spacebar or ENTER to page through, but you can search, scroll by lines, and a lot of other useful things.
This sounds simple but you'll quickly find that a lot of these old terminal tools had to solve surprisingly complicated problems.
Project 1: Log Analyzer
Project Data Sheet |
---|
The log analyzer is a classic beginner project that most people can attempt soon after learning how to process files, process strings, and work with lists. |
Starting Difficulty |
Beginner, basic programming skills required. |
Longterm Difficulty |
Ramps up pretty quickly especially if you get into analyzing logs across many machines. |
Related Topics |
Parsing, Distributed Computing, Event Sourcing, CQRS, Statistics, Data Analysis, Domain-Driven Design, Databases, DevOps |
Examples and Related Projects |
Broadway, Prometheus, EventStore, Commanded |
Starting Resources |
https://github.com/leandrocp/awesome-cqrs-event-sourcing |
Discussion
To get started with a log analyzer project just find some logs from a web server or similar, then try to collect simple statistics. This is something just about anyone with basic programming knowledge can do, and in the process of writing this you'll probably learn more about the language you're using.
Once you have a simple log analyzer going you'll want to start pushing it as far as you can. You can either keep evolving one project, or start new ones for each phase. Your eventual goal is a distributed logging and reporting system, but keep in mind this is a massive goal. If you try to do that first you'll fail (and I'll refer you to my talk about gifted perfectionists).
Key Features
- Initially just process a simple log and produce some basic stats.
- Give this tool a nice command line interface and improve the stats and logs it can parse.
- Once you have something working on one machine start working on getting it to aggregate across multiple machines. This is a BIG LEAP.
- It'll help if you can setup up an existing system across mutliple machines to see how the existing projects work.
- With aggregation across multiple machines working you'll then focus on analysis, efficiency, and presenting the data.
These features are open ended so that you have room to approach the problem how you want. These are also multi-year goals to get you into the entire world of distributed logging. Trust me, this is a big long project that branches into a ton of topics.
Project 2: Automated Web Scraper
Project Data Sheet |
---|
Web Scraping is a major component of almost all AI today. How do you think ChatGPT learned? Magic? |
Starting Difficulty |
Beginner, basic Python or JavaScript skills. Other languages possible but Python and JavaScript have lots of good stuff here. |
Longterm Difficulty |
Starts off pretty easy but over time you have to deal with a cat-and-mouse game to evade all the various ways scrapers are blocked. You can also get very deep into machine learning to parse resources, parsing in general, and statistics. |
Related Topics |
Pen Testing, Computer Security, Data Analysis, Data Munging, Evasion Techniques, Parsing, Web Development, Automated Testing, Statistics, Machine Learning, Fuzzing, SEO, Site Reliability |
Examples and Related Projects |
Scrapy, Beautiful Soup, Common Crawl, Pyspider |
Starting Resources |
awesome-web-scraping awesome-data-engineering |
A basic web scraper is not hard to make at all. You pick a website you want to scrape, grab Scrapy or Playwright, and try to extract some data from the site. "Data" could be anything, but you probably want to start with a simple site that has actual data.
Government websites are good, but be warned: Some governments require you to ask permission first before you try to scrape them. Be sure to check before you scrape a government site outside of the US. In the US all information is supposed to be public unless it's been classified, so as long as you're getting data that doesn't require a login you're should be fine.
Once you have some data flowing from a website you'll want to turn it into an automated process. Your goal with this is to get the data about once a day, then produce a report. A good example would be getting some numbers from a government website each day then sending you an email with a summary.
This then opens up a whole range of topics related to computer security and testing. For example, once you have a working scraper you'll run into how sites detect scraping, how Google codes Chrome to make detection easier, how you need proxies to jump around IP addresses, and how to evade detection with various techniques. You can then get into the other side of the game and devise methods for detecting scrapers, which gets into computer security and other fun stuff. You could also go a whole other direction and start learning automated testing and fuzzing techniques. Then finally there's even a whole branch of scraping that's related to SEO analysis.
Key Features
For your first automated scraper the requirements are simple:
- Get it grabbing some data off some website.
- Make it email or generate a report once a day automatically.
After that you'll want to see how far you can push this scraper, on the data collection and the reporting side. Key features of your big goal are:
- Caching! You've got to implement good caching so you're not thrashing the target site too much. You might want to check out squid for an initial idea.
- Nice Teporting If you're doing email reports check out MJML for a nice way to do HTML emails. If you're doing HTML then look at various charting and reporting systems.
- Logging and notifications. No point in making an automated scanner if it gets blocked and stops working.
- Scalability If you're hitting a few sites then no big deal, but you may eventually get to where you have to scale out your scraping activities, so how do you coordinate this?
- Database I'd say don't roll your own. You can use SQLite3 to get started, and then when you need distributed access look into PostgreSQL.
- Ban Evasion How are you going to avoid being detected? It's a very interesting topic, but of course don't break the law. I'm pretty sure trying to evade a ban on a government website will get you into trouble, so just contact them and get permission.
Project 3: Chat Server
Project Data Sheet |
---|
Chat servers are relatively easy to write in a naive way but quickly become very complex network protocol servers. |
Starting Difficulty |
Junior programmer, either knows or wants to learn network protocols. |
Longterm Difficulty |
You can go absolutely insane with this one, eventually getting into distributed computing, network efficiency, parsing, and even delve into AI. |
Related Topics |
Network Protocols, Parsing, Cryptography, Distributed Computing, Computer Security |
Examples and Related Projects |
UnrealIRCd,Jabber, Prosody, Matrix, netcat. |
Starting Resources |
Beej's Guide, Effective TCP/IP Programming by Jon C. Snader |
Chat servers used to be all the rage back in the day, but these days most people just text each other or use a few services like Discord. That doesn't mean writing you own chat server isn't fun and educational. You'll probably learn the most about network protocols and distributed computing from this project than any of the others.
At first your chat server will probably be really stupid. You can connect to it with netcat and type lines to yourself. That's still a huge leap and once you get that working you can add in more chat server features. You'll probably need to write a client, then a protocol for both, and what about federation? Is your chat server only working on one server or do you have failover?
Then you run into the big part of the project: Cryptography. I highly recommend you don't roll your own here, but look at libraries that implement encrypted connections or even just go SSL. However, you'll then run into the can of worms that is group chat encryption. If you think about it, encrypting a group chat is...kind of dumb. Anyone who wants to read the messages in an open group chat just...joins the group. If you want to encrypt a group chat you'll then step into a very complicated world of shared cryptographic keys, key revocation, and so many other gnarly bits.
Key Features
- Actual TCP/IP Protocol. You could easy do a chat server as a web application but that's not the point of this project. This project is all about learning how to make protocols, so you'll want to make your own custom protocol.
- Make it Hackable First. One of the biggest flaws of protocol design until recently was Postel's Maxim which was perverted into an excuse to make lazy servers that were easily hackable so that the protocol was "human readable." One way this lead to hacks is most protocols until recently were text based and line terminated, usually with a
\n
(newline) or a\0
(null). We know now that this is incredibly stupid and leads to massive hacks and horrible inefficiency, so I want you to first do it this way. Yes, I want you to make a hackable server that waits for a line terminator and then figure out how to hack it. - Fix the Hackable Version. You'll then want to switch to a "length specified" protocol. What this means is that your protocol explicitly give the length of every message in a confirmed fixed length format before sending any data. What this does is allows your server to properly allocate the correct size buffer, but also allows your server to immediately reject any messages that are too large. The line terminated protocol (hackable version) has the problem that a server has to completely read all the data before knowing that a message is too large. It also has to continually and correctly grow buffers to handle the space, which is a proven way to have a buffer overflow.
- Client and Server: Since you're doing a custom protocol you'll eventually need a custom client. This will get you into the world of GUI and TUI (Text UI) interfaces. I recommend you do a TUI first if you can, otherwise, make a GUI.
Project 4: Tiny Programming Language
Project Data Sheet |
---|
Learning how a compiler works by implementing one is probably the best way to boost your programming skills in any area of computer science. |
Starting Difficulty |
Junior programmer who knows, maybe 2 languages. Can use any language. |
Longterm Difficulty |
You could probably spend your whole life studying compilers, and many people do. |
Related Topics |
Parsing, Compilers, Programming Language Design, Assembly Language, Machine Architecture, Type Theory, Network Protocols, Education, Data Structures and Algorithms |
Examples and Related Projects |
Lemon Parser, Ragel, Lark, c500 Tiny C Compiler in Python, Tiny Python Interpreter, Elk Tiny JavaScript Interpreter, Microvium |
Starting Resources |
Crafting Interpreters, Parsing in Python, Writing a Compiler in Go |
If you want to get extremely good at programming, then implementing a simple compiler is the very best project. I predict that after this project you'll be able to learn most programming languages quickly and won't be afraid of them anymore. You'll also have a deeper understanding of the computer you use and why some languages (<cough>
C) have massive security holes baked into them.
But, doesn't that mean writing a compiler is hard? No, because compilers are incredibly logical and straight forward. There's no hand waving BS about "story points" and "object this class that." A compiler is a logical progression of: lexer, parser, semantics, generator. You write each one in mostly in order and they mostly fall into place as a consequence of the previous one. That means you can focus on each piece on its own to study how it works.
A simple compiler also can start out as a really simple interpreter. The difference between an interpreter and a compiler is this:
- An interpreter creates a structure called an Abstract Syntax Tree (an AST) and run that directly by walking it, effectively "interpreting" the language constructs to implement computation. Interpreters are usually slower than compiled languages, but have a lot of portability and usability advantages.
- A compiler also creates an AST, but instead of directly interpreting that AST it processes the AST to create a byte code for a machine. You then hand that byte code (which is what's in your
.exe
files) to a CPU and it's the CPU that runs it.
The simplest way to explain this is, "An interpreter is where the interpreter runs your code directly itself without a second step, but a compiler converts your code into something else for another system to run." This means that languages like Python are actually compiled, not interpreted because Python produces a bytecode in the .pyc
files and you can disassemble python bytecodes.
The reason I'm telling you this is you can write an interpreter for a really simple language yourself, and then turn that into a more advanced compiler later. If you stick to a language like old school BASIC you can get it working even with minimal knowledge.
Another advantage of writing your own little interpreter is that it turns programming languages into real things, not magical mysteries. I think people imagine interpreters and compilers to be difficult because they're taught programming languages in a way that treats these languages like they're magic. You just type the right incantations and hopefully things work right. After you create just one interpreter you start to think about languages as realistic logical systems you can understand.
Key Features
Your initial little interpreter should be able to process a simple language like BASIC. Don't go out and design your own language. Just pick a subset of an existing one to process. The things you're looking for in this first interpreter are:
- Basic printing and inputs.
- Most basic math operations.
- Simple control structures like
if-statements
and loops, if the language has those. If it's BASIC then most of the GOTO and other constructs. - A lexer that converts the input into tokens.
- The tokens are then fed to a parser, and you can use a Recursive Descent parser as those are easy to write by hand for simple languages.
- This parser should produce an AST, which is a structure you can walk to perform the computation.
- Finally, you have an interpreter which walks this AST, keeps track of variables, does the commands, and produces output.
This is a larger project then many of the others but it's a logical series of steps to make it. In fact, each step is something you can create and test on its own. Your interpreter can be fed custom AST trees to test it. You can create unit tests for all of the AST elements on their own. The scanner can be fed a ton of input text to test it. You can create custom lists of tokens to feed to the parser. Each piece is on its own so it's a good methodical project that anyone can attempt.
After you get a little interpreter working you'll want to either push that to the moon and make it as complete as possible. Or, convert it into a compiler for a target machine. You may even want to write your own little bytecode machine to learn about that. Realistically this is a long but highly rewarding project for anyone.
Project 5: Text Only Roguelike
Project Data Sheet |
---|
This is my current obsession and I have to say I've learned more interesting algorithms in a short time than I've learned in any business application I've written. |
Starting Difficulty |
Junior Developer, any language but C++ or C# to get into gamedev. |
Longterm Difficulty |
Rogue seems like a simple game but it's a real gateway drug into so many Game Development topics. |
Related Topics |
Game Development, Computer Graphics, AI, Algorithms, Being a Cool Mofo |
Examples and Related Projects |
Umoria, Dwarf Fortress, Nethack, Brogue |
Starting Resources |
RogueBasin, RogueElements |
This project was actually the impetus for me writing this essay. I had started working on a little roguelike to learn more game dev techniques and I realized that this one little text based game probably had more education in it than I've had in any project for decades. I'm learning everything from Entity Component System to how collision detection can use Morton codes by interleaving the bits in a byte? What? That's cool as hell.
What I'm finding makes this project so fun an educational is it focuses on one key aspect of game development at a time. You don't have to know how graphics, rendering, 3d space, or physics work. You "collision detection" is mostly just "is this cell occupied?" That means you are free to focus on learning other aspects of game development like the algorithms, data storage and management, saving, resource management, combat systems, game play, music and sound, and efficiency.
When you're trying to learn something really vast like "game development" it helps to strip the topic down and then focus only on one aspect at a time. When I was learning to paint I didn't just start painting (although that'd probably work). I first learned to draw, then I learned how to mix colors, and then I applied those to the problem of painting what I saw. Painting requires drawing, color perception, and the actual act of painting, so focusing on each topic separately helped immensely.
Writing a little roguelike does this for game development, and Rogue style games are still a ton of fun to actually play. It's going back to an older time when all we had was text terminals, so it adds this constraint that makes you focus on other aspects of the game. Since you can't impress people with flashy graphics you have to work on things like game play, mechanics, and narrative to make a compelling game.
Key Features
Your first little roguelike game doesn't have to be complex at all. At the time of this writing I've got a little display and two enemies that chase the player around. That's what you should aim for:
- A simple map that is generated with a couple rooms. Map generation is a fun challenge so look up Binary Space Partitioning.
- Dijkstra's Algorithm for maps is incredibly important and a fun little algorithm to figure out. I decided to take this implementation in Python and rewrite it in C++ for my game, but you could probably make a nice little project out of figuring out this algorithm.
- Entity Component System is probably overkill for a little rogue game, so start off with a "list of objects" and iterate over them. Still, if you want to use ECS just realize it's just a SQL database. Entity is just a table column id. Component is just a database table. System is just queries.
- You'll need either a curses library, or some other TUI library. I really like FTXUI for C++.
- You don't need anything more than one player, some hitpoints, and a monster to escape. Get that working then start thinking about things like weapons, combat, movement etc.
- For enemy movement you'll definitely need Dijkstra's Map. You may also want to implement a simple Line of Sight system. The easiest one is "are they in the same room?"
Once you have that all working--and it'll take a while--you're then free to go nuts with it. In my Roguelike game I'm going with the theme "how far can you push text?" I don't even really have combat mechanics yet as I'm exploring the algorithms and how to use text in creative ways in the game. Just keep working on it and have fun with it while using it as a vehicle for learning the algorithms you'll need for other projects.
Project 6: 3d Scene Renderer
Project Data Sheet |
---|
It may seem complex--because it is--but learning how to render a 3d scene is approachable with some effort. There's even Academy Award winning books to teach you how to do it. |
Starting Difficulty |
Junior developer, probably should know C++. |
Longterm Difficulty |
Learning how to render 3d scenes opens up a whole world of computer graphics and game development. |
Related Topics |
Game Development, Computer Graphics, Data Structures and Algorithms, Linear Algebra, Demoscene, 3d Art. |
Examples and Related Projects |
Filament, Irrlicht, Ogre3d, libigl, Wicked Engine |
Starting Resources |
Learn OpenGL, Physically Based Rendering, Awsome Graphics Libraries |
In the complete inverse of the Roguelike project we have learning how 3d rendering works. I'll be honest and tell you that this is on my list of projects to do next, so my knowledge here is not too good. However, two resources that have been repeatedly mentioned as starting points are Physically Based Rendering and learnopengl.com.
Since I don't actually have this knowledge yet I'll tell you how I plan on learning this when I start the project:
- I first started in 2D with the project SFML since many of the concepts carry over into 3d. SFML has things like textures, transformations, and shaders so starting there works.
- I then wrote a dumb little game called Turing's Tarpit that is just a single panel in SFML and C++. The idea is to learn how 2d coordinates and rendering works.
- My next step is to then go through either PBR or learnopengl.com and make all the code work. That's the key to learning from other resources. Even if you don't quite understand what's being said, you can learn a lot about the topic through the code. So, type it all in, get it working, try to understand it, then continue.
- After that I'll probably try out a few rendering engines to see how they work, or just finally jump into making a game with something like Unreal Enginer or Godot.
I will say a big inspiration for me finally learning 3d rendering is tokyospliff's streams. He just sits on his couch coding directly in C++ and OpenGL/Vulkan to make a game. He's recommended learnopegl.com many times, and says that's how he learned years ago when he didn't even know how to code.
Project 7: Deluxe Data Munger
Project Data Sheet |
---|
Data munging or data wrangling is the process of turning data sources that are hot garbage into gourmet meals. |
Starting Difficulty |
Beginner, any language would work. |
Longterm Difficulty |
Data Munging eventually turns into forensic document analysis, and even has elements of computer graphics and and AI. |
Related Topics |
Data Analytics, AI, Statistics, Archeology, Law, Politics, Parsing, Image Analysis, Forensics. |
Examples and Related Projects |
PDF table extract |
Starting Resources |
awesome-pdf awesome-data-wrangling awesome-data-engineering |
This project is probably the easiest one to get into if you're just starting out, but has a lot of potential to give you topics for a few years. "Data Munging" is the act of turning terrible garbage data into usable data. You'll find things like tables in terrible PDF files, poorly formatted CSV files, broken Excel data, and badly formatted databases. Then you write a program to load them, parse them, clean them, and finally store the important data somewhere useful.
The "deluxe" part of this project is where the real action is. Initially you'll just make a data munger for a specific data set. You could even combine this with the Web Scraper project earlier. Once you have one data source working, you'll then want to attempt other sources until you realize something: You could make a data munging platform that can import many different formats, clean them, and send them to a single data repository.
This long term project involves many different possible directions. Maybe you get into the automation aspects of it, making it easy to define data pipelines. Maybe you get into the extraction aspect of it where you explore AI, computer vision, and parsing. You could combine this project with many of the other projects on this list. The Programming Language and Web Scraping projects are good related projects.
Key Features
Initially your data munging could just be a little program that analyzes a PDF data source or similar gross data. Keep it simple with just these features:
- Can ingest a target data source and reliably extract the features.
- Your solution should store the data in a database like SQLite3 to make it possible to analyze after.
- It should do correct logging and reporting to catch changes in the source data causing errors.
- You definitely want to create automated tests for it to make sure it keeps working.
Once you have this simple program working you'll want to tackle the problem of processing other random sources of data. You then have a lot of design issues to solve. How do people specify what data to get, how it's transformed, and where to put it? Do you add reporting or is that outside the scope of your project? How advanced is your extraction system? Does it require code to use or does it have a nice GUI?
Project 8: Docker Replacement
Project Data Sheet |
---|
Docker is ripe for disruption, so why not try to take it out? You probably can't but making your own will teach you so much. |
Starting Difficulty |
Junior developer, probably Zig, Rust, or Go would be good for this. |
Longterm Difficulty |
You could spend quite a lot of time exploring the automated hosting space, but this branches out into topics like distributed computing and computer security as well. |
Related Topics |
Computer Security, Network Protocols, DevOps, Hosting, Automated Deployment, Containers, System Administration, Distributed Computing |
Examples and Related Projects |
bocker |
Starting Resources |
What is Docker Docker Command Line Options LXC and LXD |
The best way to learn how something works is to make your own version. If you're someone who's interested in modern Devops and projects like Kubernetes, then making an alternative to Docker is a great way to learn how Docker and similar container systems work. It's also not as complex as you think given someone made a Docker replacement in bash.
This project is pretty easy to specify for features: Just copy Docker. However, I will give you a few important things to understand:
- Docker's main flaw is that it's trying to be too many things at once. It's simultaneously a build tool for deployment scripts, a automated deployment system, and a container system. For example, having the layered file system you can roll back is great while you're working on the deployment, but having that same layered system on every production deployment is a massive waste of disk and performance. There's a reason a bunch of Docker scripts are just tons of lines separated by
\
characters. - Because of this you could focus on each of these components separately to approach creating your own system. For example, while I'm developing a container I don't actually need any containers or deployment. You could work on a container development tool that doesn't really use containers or any automated deployment, kind of like a programming language compiler.
- You could also leverage the LXD to get an initial solution to the containers and deployment part, so you only need to focus on the automation and deployment development parts.
Project 9: Arduino Bread Board
Project Data Sheet |
---|
I literally found a little plastic box full of an old Arduino project kit I never used. If you're interested in physical devices and interacting with the real world this is the project. |
Starting Difficulty |
Junior Developer, Python, Ruby, C or C++. |
Longterm Difficulty |
Learning how to control a bread board with an arduino opens up a ton of Electrical Engineering projects, and even can curve into making electric guitars. |
Related Topics |
Electrical Engineering, Embedded Systems, Electronics, Soldering, |
Examples and Related Projects |
ArduinoGetStarted/led |
Starting Resources |
Awesome Arduino, 15 Arduino Projects |
One of the most fun things I learned years ago was how to build electric guitars. Electric guitars are fairly easy to assemble from parts, but have slightly complicated electronics. The electronics are from the 1950s though, so you aren't working with any kind of chips like an Arduino. Just soldering wires to potentiometers and magnetic coils. Electronics are fun, but they also turn into really amazing long term learning projects and give you the ability to interface your code with real-world things.
One of my students started off with just an Arduino and a breadboard with a little LED on it. After that he started making more and more complex circuits like motion detectors. Since I haven't actually done this project yet, I'll just quote him as saying, "Making a little LED light is kind of the 'hello world' of Arduino hacking."
For this project you'll want to go get an Arduino starter kit that comes with an Arduino board, a bread board, and some components to play with. I actually found an old box while cleaning my garage that I forgot I purchased, so I'll try this project out myself and update this section when I've done some tinkering with it.
Project 10: Video Website
Project Data Sheet |
---|
Displaying a video on a web page is fairly easy, but displaying a video efficiently and with improved user experience is a nightmare. |
Starting Difficulty |
Junior developer, any web development language but I think Go would be the king of this. |
Longterm Difficulty |
You can get into everything from automated hosting to efficient video and audio compression. |
Related Topics |
Distributed Computing, Video and Audio Compression, Network Protocols, User Experience, Web Development, Hosting, System Administration |
Examples and Related Projects |
peertube Pion WebRTC WebTorrent |
Starting Resources |
awesome-pion |
This project is one that may seem really easy--and for a beginner it's a simple start--but it will quickly ramp up and branch out into User Interface design, Web development, and hosting. You see, while you can just put a video on a web page with the <video>
tag, that actual functionality of that tag is dog water bad. Let's just go through some of the issues:
- You have to manually write your own overlays to "soft load" a video. What I mean by this is if someone visits a page the video tag will try to load some or all of the video. This slows down the page load and thrashes your web servers, so what every video hosting service does is put a graphic overlay where the video will go. If you don't click on this overlay then the hosting company has saved a ton of bandwidth and their page loads fast. If you do, then they swap out the graphic and load the video. The question to ask is...why doesn't the damn browser just do this for you?
- The default video tag has a terrible UI. In my own experience I had to put a massive play button icon in the overlay just so people knew they had to click on it. The video tag also give a lot of controls for things like changing speeds, no information on what speeds are good, and has no almost no ability to adapt the video quality to the user's screen size.
- This tag also has no support for most methods of distributing video load across servers, which is absolutely ridiculous given HLS exists. Safari supports it, but if you want reliable HLS support you have to use a JavaScript library. I'm not kidding. It's that stupid. You actually have to code your own HLS support if you want to spread the load out.
- Overall the
<video>
tag lacks almost any features necessary to host videos yourself. Weird. I wonder why a company that runs a massive video hosting website would do their very best to make sure nobody else can efficiently host videos?
This is the problem you'll be tasked with solving. At first you will work on a little website to host some videos. Do your best to make it look nice and get some videos up there.
Once you get it working then turn your eye to the problems I describe above, and start to think about the hosting angle too. How are you going to encode these videos so they play well and save on bandwidth? Are you going to use AV1? Well, better prepare yourself for 2 days of encoding to save 20%. Is your site allowing user submissions? How do they do uploads?
This one project will open up tons of web backend and frontend development education, and it's related to projects on this list like Docker Alternative, Automated Web Scraper, Log Analyzer, and Chat Server.
A Warning for the Gifted Perfectionists
One group of people who will struggle with this exact topic is the "gifted perfectionists." These are typically people who were told they were gifted as a kid, and the result of this is they became perfectionists who are perpetually afflicted with fear of failure and a desire to impress people in positions of authority.
Sorry to be so harsh, but the combination of these two maladaptive behaviors--fear of failure and a desire for praise from authority--means you will sabotage yourself in two possible ways:
- You will take my simple project ideas and turn them into insane projects nobody can complete. You'll do this because you hope that I'll praise you for being so smart for trying to do something insane like that (I won't), and to absolve yourself of failure because it was "just so difficult nobody could do it."
- You will research each of these projects forever and never actually make anything. Research and memorization is safe with no real failure mode--and you can easily impress people with your knowledge on IRC and Stackoverflow--but memorizing things about programming is not the same as actually making things with what you learn.
If you find that you attempt either insanely impossible goals, or things that are way too easy, then I'm probably talking about you. If you find that your main motivation in doing anything is to receive praise from people--especially people in positions of power or that you respect--then I'm definitely talking about you.
The way to get out of this trap is to do the following:
- Don't tell anyone about your ideas or your progress until you actually have something to show. This prevents you from seeking praise from authority and keeps you safe from being derailed in your attempt.
- Write down all the crazy things you're totally going to do and then strip out 90% of it. Just assume your brain is trying to sabotage your chances at success and strip it back to reality.
- Adopt an attitude of "continous improvement" over "pass or fail." These topics are great because they're basically never ending trains of study so the only way you fail is if you stop trying to learn them.
- Set small goals and things to accomplish each day rather than big dreams about massive projects that will impress people.
- Understand that the creative process is messy in the middle and don't stop until you finish. I believe that perfectionism is just a misunderstanding of the creative process, and that perfectionists believe you create something by keeping it perfect and pristine the entire time when the reality is that creativity is messy and chaotic in the middle of the process. Embrace the chaos and keep telling yourself it's not done yet.
- When it is done give yourself an honest critique, then show it to someone else you trust who will give you honest feedback. Even if you think it's crap ask someone's opinion anyway so you rebalance your perception of your skills. You may be too hard on yourself, or you may be delusional. Just take in the feedback and then do it all again until you get good.
Most importantly, you'd do yourself a huge favor by trying to shed this belief that you are are gifted. Even if it's true there's a high probability that it's holding you back and making it harder for you to learn new things.
"But a Famous Programmer Said..."
The next thing to talk about is the trend over the last 10 to 20 years of famous programmers using their platform to stifle other programmer's creativity. I myself have been guilty of that a few times in the past, but these days I think it's abhorent behavior.
If anyone tells you that you are not qualified to attempt these projects, or has said that "we already have enough of those" then just ignore them. Every programmer influencer who says this is really just trying to keep you from potentially learning that you don't really need them. You see, if you learn how to make your own React then you don't need Youtube videos by douchebags telling you what you should think about React. You know from your own experience.
This is your learning experience, not theirs so learn what you want and enjoy the experience. And if you ever get some blowhard yelling at you send them to me. I'm itching for a fight about this.
About Publishing Your Work
You don't have to put your projects out there but here's some advice if you would like to:
- AGPL it if you want to put it on Github.
- Host your own Forgejo site and put your stuff there to make it copyright.
- You can alter the list of licenses to add a Copyright license that's basically empty or just with your Copyright statement.
- Share your code with your friends that way and you are totally allowed to do this.
You do not have to license your software! Your software is already Copyright by default, and you are also allowed to post your code online without the threat of being sued and without worrying about someone stealing it. Copyright says they can't steal it, but you also have a right to Freedom of Speech. If you were required to license your code before releasing it then that would create a chilling effect on your speech as a programmer. People who aren't comfortable with licensing their software would not exercise their right to free speech, so requiring licenses is intended to limit your right to speak through code.
Is code speech? Damn straight it is, and that's from the Supreme Court in Bernstein v US Department of Justice. Given that code is speech, then saying that code without a permissive license is vulnerable to lawsuits or exploitation would make code the only form of speech that is restricted. That's not true at all, so you can publish all the code you want and don't need to license it at all.
Think about it this way: Photographers, Musicians, Writers, and Artists post their works online constantly, without licensing or warnings. They also post horribly heinous garbage and that's their right, even if you or I disagree with it. The only time they "license" their works is when they publish on a platform they don't own, and that license is not permissive. They license their video directly to Youtube and there's strict requirements in the license. But if you were a writer with a blog and you owned the hosting then you do not have to license anything you write. You also don't have to make any statements about whether people are allowed to read it because the act of publishing is your right to express yourself and it's implied that you want people to read it. That's because fair use exists to allow other people to read, share links to, and comment on what you create.
Now, tell me, why is code suddenly so different that we all think we need to license it? Code is speech, just like paintings, poems, ballads, and cyanotypes. Is it because someone might "accidentally" put your code in an MRI machine and kill someone? Guess what, you didn't say they could use the software that way and you didn't put your code in the MRI machine. It would be that programmer's fault, not yours.
But, there's a simple fix to this irrational "but what if I get sued because someone puts my video game about evil puppets into a heart monitor and people die?!?!?!?!" Just put a stupid disclaimer on it that says they can't sue you. That's it. No more fear of lawsuits so now you can publish your work just like that guy who wrote the Anarchist's Cookbook which told people how to make bombs.
Finally, let's say there is a real threat that you'll be sued if you dare to exercise your freedom of speech and publish your code. Ok, then why do you have to add a permissive license that lets Amazon steal it? You see the problem here? You don't have to give your code away just to avoid being sued. You only have to slap a disclaimer on it and that's it. You can publish it, others can look at it, because code is speech and that's how all other speech works.
All this is to say, stop worrying about licensing and just make stuff. Put it online. Express your freedom of speech because code is speech just like photos, music, movies, writing, and art, and any demands that you license your rights away so trillionaires can exploit you is ridiculous. Keep it copyright until you know you want to license it for a specific reason that requires a license.
And if a lawyer tries to tell you I'm wrong just ask them, "Given code is speech according to our highest court then why is software the only form of speech that requires permissive licensing to publish it?"
Or, send them to me, I'm itching to fight someone on this too.
More from Learn Code the Hard Way
Very Deep Not Boring Beginner Projects
A list of projects that anyone with basic programming knowledge can do in any language, but have a lot of depth and aren't boring to do.
C++ Is An Absolute Blast
Where I try to explain why I feel like C\+\+ is so much fun, and to correct some misinformation.
Just Use MSYS2 You Say?
Debunking idiots on HackerNews who think a total beginner can just 'Use MSYS2' on Windows.
Solving the Worst Problem in Programming Education: Windows
I've created a set of install scripts for Windows that helps people install many different programming languages using standard tools.