After creating so many things I have realized there MUST be more abstraction between parsing and code execution than what I currently have, which leads to my main question(s).
Not really. In a typical industrial-strength production-quality high-performance language implementation, there usually are many more stages, but that is by no means a necessity. Simple tree-walking interpreters such as MRI Ruby, for example, really just lex, parse, then walk the abstract syntax tree and execute a snippet of code for every node visited. That's it. Even simpler line-based interpreters don't even build an AST.
My first question is, this. What exactly is the job of a language virtual machine, like for example the JVM […].
The most important thing to realize is that a VM is nothing special. A virtual machine is anything which completely abstracts one API from a different API. That's it. For example, in object-oriented programming (at least in the form envisioned by Alan Kay), every object is a virtual machine.
More specifically in the context of languages: every language is a VM and every VM defines a language.
Really the only difference between a programming language and a VM instruction set is intent: the syntax of a programming language is designed to be easily read by humans, the semantics are designed for elegantly expressing complex problems. The syntax (format) of a VM instruction set is designed to be easily parsed by machines, and its semantics are designed to be easily interpreted (or compiled) and also easy to be compiled to.
But again: a VM instruction set is just a programming language like any other, and it is executed exactly the same way as any other programming language: by either interpreting it, or by compiling it to a different language that you already can interpret.
What does it do? Does it actually sort of emulate hardware and execute low level instructions on the emulated hardware? Or does it simply provide an abstraction and execute lower level managed instructions on the system?
That depends on what you want the VM to do. The LLVM instruction set, for example, is designed to be a machine-independent machine code, if you will. Its goal is to be easily compiled into efficient machine code. So, it tries to be as low-level and close to current mainstream CPU instruction sets (x86, AMD64, IA-64, SPARC, MIPS, ARM, etc.) as possible without tying it to a specific CPU instruction set.
The JVML instruction set OTOH was designed to be easily interpreted, and as such is much higher-level than the LLVM instruction set. The CIL instruction set is equally high-level as the JVML, but it was designed to be compiled, and so makes some different choices.
Second question. What exactly is byte code? Is it some sort of machine code for the virtual machine? Or is it the human readable possibly assembly like instructions,
Byte code is simply the name for a language where instructions are encoded as bytes, instead of as text. That's it.
Note that many languages which are erroneously referred to as "byte code" actually aren't. For example, on the CLI, instructions are encoded as ints, not bytes, so CIL is an int code, not a byte code.
and are these instructions directly executed or somehow compiled to native code?
Either. Both. You decide. The first one is called "interpretation", the second is called "compilation". By the way: you don't need to compile to native code, you can compile to anything for which you have an interpreter or compiler. For example, the Fantom VM compiles its instruction set to either JVML or CIL, depending on whether it is run on the Java Platform or the CLI.
Third question. How would I go about implementing this? I consider myself a competent programmer and don't need step by step instructions by no means, just an overview of what exactly I need to be doing, and how low level this is exactly.
A VM instruction set is a language just like any other language. You implement it just like any other language:
- semantic analysis
- type inference
- type checking
- code generation (for a compiler) or code execution (for an interpreter)
The parsing stage is usually trivial, because VM instruction sets are designed to be easy to parse. Type inference and type checking of course only make sense if the VM instruction set is typed. Optimization is optional, if you don't want to build a high-performance VM. OTOH, if you do want to build a high-performance VM, then optimization is where you are going to spend 99.999% of your development effort.