There's the very big problem stopping me from continuing the course. There are some things I added to the OS, that were tested and worked perfectly. But when I was on lecture 83, after I compiled all code and tested it in QEMU, there were very strange things on the screen. I debugged it with GDB, and figured out, that the code is not going past the line 100 ("if (filesystems[i] != 0 && filesystems[i]->resolve(disk) == 0)") in file.c, where is the code for VFS. (I haven't changed that file anyhow), and the QEMU screen starts showing random symbols in top left corner at this moment. I solved that issue by removing the classic keyboard driver from Makefile, but then, when I added some changes myself, the problem appeared again, like if I exceeded some sort of kernel size bounds. Is that a memory error, or what can cause this issue?
P.S. I watched those random symbols appear for some time, and I saw that those symbols are in extended-ASCII order, and changing colors randomly, but in order too. Then, after a fixed period of time, the kernel panic message shows, but the actual panic message almost entirely composed of non-printable characters (I have no panic() call, that could print this).
I know it may seem like something random, but that's an actual issue.
Hello,
Apologizes for the delayed response I have been off work for new year break. Have you tried downloading the source code at lecture 83 and seeing if that works. Many people have completed this course before you so its more likely you made a mistake. You can then compare the source code on lecture 83 with the source code on your own repository.
Let me know how that goes and I can be of further assistance.
Many thanks,
Dan
I did it, and it didn't help me. As I said, I removed the file from Makefile, and it worked, but then I just added several prints, and it happened again. And again it works without those prints.
Also, it seems like it starts happening, when kernel .bin file exceeds 65536 bytes (16-bit bounds). Can it be a bootloader issue? And is there any way to fix it without splitting the kernel?
Your saying that you downloaded the actual source code? And rebuilt it and it still did not work? That code has been ran by over 3000 students, in which case it must be something else going on, is your compiler 64 bit or 32 bit?
Can you please upload some screenshots of what is going on here: https://imgbb.com/ and send the link
Also please provide your linker.ld file and Makefile at pastebin.com and send those links too.
It is possible your sections are misaligned in your linker script, we do align them in this course, if you can send those I will be able to assist more.
This course is designed to teach you how to create your very own multitasking operating system and kernel from scratch. It is assumed you have no experience in programming kernels and you are taught from the ground up.
Real Mode Development
Real mode is a legacy mode in all Intel processors that causes the processor to start in a legacy state, it performs like the old 8086 Intel processors did back in the way.
In the "Real Mode Development" section of the course we start by learning about the boot process and how memory works, we then move on to creating our very own boot loader that we test on our real machine! This boot loader will output a simple "Hello World!" message to the screen and we write this boot loader in purely assembly language.
In this section we also read a sector(512 bytes) from the hard disk and learn all about interrupts in real mode and how to create them.
This section gives you a nice taster into kernel development, without over whelming you with information. You are taught the basics and enough about the legacy processors to be able to move forward to more modern kernel development further into this course.
Protected Mode Development
In this section we create a 32 bit multi-tasking kernel that has the FAT16 filesystem. Our kernel will use Intel's built in memory protection and security mechanisms that allow us to instruct the processor to protect our kernel and prevent user programs from damaging it.
This section is very in depth, you are taught all about paging and virtual memory. We take advantage of clever instructions in Intel processors to allow all processes to share the same memory addresses, this is known as memory virtualization. We map memory addresses to point to different physical memory addresses to create the illusion that every process that is running is loaded at the same address. This is a very common technique in kernel development and is also how swap files work (Those files that are used to compensate for when you run out of usable RAM).
We create our own virtual filesystem layer that uses a design that is similar to the Linux kernel. This clever abstraction that will be taught to you was inspired by the instructors knowledge of writing Linux kernel drivers in his past.
You are taught about the design of the FAT16 filesystem and how the FAT16 filesystem is broken down into clusters and that they can chain together. We then implement our very own FAT16 filesystem driver allowing files to be born!
We implement functionality for tasks and processes and write our own keyboard drivers.
In this course you also get to learn how memory management works, we implement the "malloc" and "free" functions creating our very own heap that's designed to keep track of what memory is being used. Memory management is essential in any operating system and kernel.
Let us not forget that we even create an ELF file loader, we will compile all our operating systems programs into ELF files and allow the loading of binary programs or ELF programs. ELF files contain a lot of information that describes our program for example where our program should be loaded into memory and the different sections of the program.
By the end of this course you will have a fully functioning 32 bit multi-tasking kernel that can have many processes and tasks running at the same time. You will have a working shell that we can use as well.
Assembly language bonus
This is a bonus section designed to bring your assembly skills up to scratch should you struggle a little bit with the assembly language in this course. It's however advised you come to this course with experience in assembly language, we do use it and its important. Never the less if you want to take a chance on this course with no assembly experience then this section will help point you in the right direction so your able to take what you learned and apply it to the kernel.
Taught by an expert that has created Linux kernel modules professionally in the work place.
Requirements
Understanding the C programming language
Understanding Assembly Language
Who This Course is For
People who want to develop a kernel from scratch
What You Will be Learn
How to create a kernel from scratch
How to create a multi-tasking kernel
How to handle malicious or problematic programs in your operating system. Terminating them if they misbehave.
How memory works in computers
The difference between kernel land, user land and the protection rings that make up modern computing
Kernel design patterns used by the Linux kernel its self
You will learn all about virtual memory and how to map virtual addresses to physical addresses
You will learn how to make the kernel understand processes and tasks
You will learn how to load ELF files
You will learn how to debug disassembled machine code
You will learn how to debug your kernel in an emulator with GDB.
Daniel McCarthy is a seasoned software engineer, boasting an impressive career spanning over 14 years in the industry. Holding a Master's Degree in Advanced Computer Science from Cardiff Metropolitan University, his broad spectrum of experience encompasses everything from web development to complex compiler and interpreter development.
Daniel has honed his skills in bootloader and kernel development. In testament to his proficiency in the field, he has designed two proprietary programming languages: Craft, a general-purpose language, and Marble, a web-focused language akin to PHP. Moreover, he has successfully developed compilers for the C programming language.
A testament to his versatility, Daniel demonstrates proficiency in an extensive list of programming languages that includes C, C++, Java, x86 Assembly language, PIC assembly, SQL, PHP, HTML5, JavaScript, CSS, and of course, his own creations, Craft and Marble.
His professional portfolio also includes the development of Linux kernel modules, a task he has executed with proficiency in a professional context. Currently, Daniel is channeling his wealth of experience and expertise into the education sector, with the aim of nurturing the next generation of professional software engineers.
Ask a question
Questions (1)
Svyatoslav Matveenko
2 years ago
There's the very big problem stopping me from continuing the course. There are some things I added to the OS, that were tested and worked perfectly. But when I was on lecture 83, after I compiled all code and tested it in QEMU, there were very strange things on the screen. I debugged it with GDB, and figured out, that the code is not going past the line 100 ("if (filesystems[i] != 0 && filesystems[i]->resolve(disk) == 0)") in file.c, where is the code for VFS. (I haven't changed that file anyhow), and the QEMU screen starts showing random symbols in top left corner at this moment. I solved that issue by removing the classic keyboard driver from Makefile, but then, when I added some changes myself, the problem appeared again, like if I exceeded some sort of kernel size bounds. Is that a memory error, or what can cause this issue? P.S. I watched those random symbols appear for some time, and I saw that those symbols are in extended-ASCII order, and changing colors randomly, but in order too. Then, after a fixed period of time, the kernel panic message shows, but the actual panic message almost entirely composed of non-printable characters (I have no panic() call, that could print this). I know it may seem like something random, but that's an actual issue.
Daniel McCarthy
2 years ago
Hello, Apologizes for the delayed response I have been off work for new year break. Have you tried downloading the source code at lecture 83 and seeing if that works. Many people have completed this course before you so its more likely you made a mistake. You can then compare the source code on lecture 83 with the source code on your own repository. Let me know how that goes and I can be of further assistance. Many thanks, Dan
Svyatoslav Matveenko
2 years ago
I did it, and it didn't help me. As I said, I removed the file from Makefile, and it worked, but then I just added several prints, and it happened again. And again it works without those prints.
Svyatoslav Matveenko
2 years ago
Also, it seems like it starts happening, when kernel .bin file exceeds 65536 bytes (16-bit bounds). Can it be a bootloader issue? And is there any way to fix it without splitting the kernel?
Daniel McCarthy
2 years ago
Your saying that you downloaded the actual source code? And rebuilt it and it still did not work? That code has been ran by over 3000 students, in which case it must be something else going on, is your compiler 64 bit or 32 bit? Can you please upload some screenshots of what is going on here: https://imgbb.com/ and send the link Also please provide your linker.ld file and Makefile at pastebin.com and send those links too. It is possible your sections are misaligned in your linker script, we do align them in this course, if you can send those I will be able to assist more.