Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MVP Loader implementation #458

Merged
merged 24 commits into from
Sep 18, 2019
Merged

MVP Loader implementation #458

merged 24 commits into from
Sep 18, 2019

Conversation

roblabla
Copy link
Member

@roblabla roblabla commented Aug 27, 2019

MVP loader. Things to do before merging this PR:

  • Implement and use unmap_process_memory.
  • Various unimportant TODOs.
  • Fix clippy warnings.

@todo
Copy link

todo bot commented Aug 27, 2019

sagiegurari/shell2batch#9

SunriseOS/Makefile.toml

Lines 224 to 234 in 2ba154e

# TODO: https://github.com/sagiegurari/shell2batch/issues/9
touch external/filesystem/disk_template/bin/0200000000000000/flags/boot.flag
cargo run --manifest-path disk-initializer/Cargo.toml -- DISK.img 52428800 external/filesystem/disk_template/
'''
]
[tasks.qemu]
description = "Runs the bootable ISO in qemu."
dependencies = ["iso-release", "disk"]
command = "qemu-system-i386"
args = [


This comment was generated by todo based on a TODO comment in 2ba154e in #458. cc @roblabla.

@todo
Copy link

todo bot commented Aug 27, 2019

Write our threat models. Will require careful planning.

TODO: Write our threat models. Will require careful planning.
[Boot Configuration Table (BCT)]: https://switchbrew.org/wiki/BCT
[Package1]: https://switchbrew.org/wiki/Package1
[switchbrew wiki]: https://switchbrew.org/
[NPDM]: http://switchbrew.org/index.php?title=NPDM


This comment was generated by todo based on a TODO comment in 2ba154e in #458. cc @roblabla.

@todo
Copy link

todo bot commented Aug 27, 2019

ASLR

We should generate a random aslr base.


// TODO: ASLR
// BODY: We should generate a random aslr base.
let aslr_base = 0x400000;
let procinfo = ProcInfo {
name: name,
process_category: ProcessCategory::KernelBuiltin,
// TODO: Put TitleId somewhere in the ELF or args or something?
title_id: 0,
code_addr: aslr_base as _,
// We don't need this, the kernel loader allocates multiple code


This comment was generated by todo based on a TODO comment in 2ba154e in #458. cc @roblabla.

@todo

This comment has been minimized.

@todo
Copy link

todo bot commented Aug 27, 2019

Ask the weird animal if this will unleash the kraken.

// TODO: Ask the weird animal if this will unleash the kraken.
//const_assert!(UserLand::START.0 % (ENTRY_COUNT * PAGE_SIZE) == 0);
const_assert!(RecursiveTablesLand::START.0 % (ENTRY_COUNT * PAGE_SIZE) == 0);


This comment was generated by todo based on a TODO comment in 2ba154e in #458. cc @roblabla.

@todo

This comment has been minimized.

@todo

This comment has been minimized.

@todo

This comment has been minimized.

@todo

This comment has been minimized.

@todo

This comment has been minimized.

@todo

This comment has been minimized.

@todo
Copy link

todo bot commented Aug 27, 2019

Create memory region reservations

Memory region reservations is sort of insane in HOS/NX - especially for 32-bit. I'll figure it out later.


// TODO: Create memory region reservations
// BODY: Memory region reservations is sort of insane in HOS/NX - especially
// BODY: for 32-bit. I'll figure it out later.
newproc.pmemory.lock().create_regular_mapping(VirtualAddress(procinfo.code_addr as usize), procinfo.code_num_pages as usize * PAGE_SIZE, MemoryType::CodeStatic, MappingAccessRights::k_r())?;
let curproc = scheduler::get_current_process();
let hnd = curproc.phandles.lock().add_handle(Arc::new(Handle::Process(newproc)));
Ok(hnd as _)
}


This comment was generated by todo based on a TODO comment in 2ba154e in #458. cc @roblabla.

@todo
Copy link

todo bot commented Aug 27, 2019

Use svcGetInfo to get the address space in find_free_address

We should use svcGetInfo to get the address space in `find_free_address`. This is extremely important as the low addresses (from 0 to ADDRESS_SPACE_MIN) are not usable, but are marked as available by QueryMemory.
Here's a sample implementation for when we'll have GetInfo impl'd.
```rust lazy_static! { static ref ADDRESS_SPACE: (usize, usize) = { let addr_space_base = syscalls::get_info(Process::current(), 12, 0).unwrap(); let addr_space_size = syscalls::get_info(Process::current(), 13, 0).unwrap(); (addr_space_base, addr_space_base + addr_space_size) }; } ```


// TODO: Use svcGetInfo to get the address space in find_free_address
// BODY: We should use svcGetInfo to get the address space in
// BODY: `find_free_address`. This is extremely important as the low
// BODY: addresses (from 0 to ADDRESS_SPACE_MIN) are not usable, but are
// BODY: marked as available by QueryMemory.
// BODY:
// BODY: Here's a sample implementation for when we'll have GetInfo impl'd.
// BODY:
// BODY: ```rust
// BODY: lazy_static! {
// BODY: static ref ADDRESS_SPACE: (usize, usize) = {


This comment was generated by todo based on a TODO comment in 2ba154e in #458. cc @roblabla.

@todo

This comment has been minimized.

@todo

This comment has been minimized.

@todo

This comment has been minimized.

Copy link
Member

@marysaka marysaka left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

Sysmodule,
/// Pool of memory usable by nvidia's driver.
///
/// <Insert nvidia meme here>
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why-cant-you-be-normal-unintelligible-screeching-you-havent-experienced-40211268

if meminfo.address() < addr {
dstmem.map_partial_shared_mapping(frames.clone(), meminfo.address(), meminfo.phys_offset(), addr - meminfo.address(), meminfo.state().ty(), meminfo.flags()).expect("Can't fail");
}
if meminfo.address() + meminfo.length() > addr + size && meminfo.length() > PAGE_SIZE {
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, you're checking if there's an after part, and remapping it as is. But why the meminfo.length() > PAGE_SIZE condition ?

Copy link
Member Author

@roblabla roblabla Sep 10, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm...


if size > MAX_ELF_SIZE {
error!("Why is titleid {:016x} so ridiculously huge? It's {} bytes.
Like, seriously, stop with the gifs!", titleid, size);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

😂

giphy-1

// BODY: a vector of frames) and comparing them.
// BODY:
// BODY: We could do something similar by iterating over the Mappings and
// BODY: checking if their Frames + PhysOffset are equals.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This syscall is really shitty.

I don't get why it didn't simply create a handle of some kind, and you could simply unmap the handle as a whole.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤷‍♂ I don't find it all that bad. Just maybe a bit too flexible.

I do hate that closing the handle doesn't automatically unmap the frames though.

@todo

This comment has been minimized.

@todo
Copy link

todo bot commented Sep 11, 2019

UnmapProcessMemory: Verify that the src page list == dst page list.

In UnmapProcessMemory, we don't ensure that src_address is correct, that is, we don't check that the frames in the dst match the frame in the src. HOS/NX does this by building a PageList (essentially a vector of frames) and comparing them.
We could do something similar by iterating over the Mappings and checking if their Frames + PhysOffset are equals.


// TODO: UnmapProcessMemory: Verify that the src page list == dst page list.
// BODY: In UnmapProcessMemory, we don't ensure that src_address is correct,
// BODY: that is, we don't check that the frames in the dst match the frame
// BODY: in the src. HOS/NX does this by building a PageList (essentially
// BODY: a vector of frames) and comparing them.
// BODY:
// BODY: We could do something similar by iterating over the Mappings and
// BODY: checking if their Frames + PhysOffset are equals.
// Unmap.
while size != 0 {


This comment was generated by todo based on a TODO comment in 9e93f28 in #458. cc @roblabla.

@todo
Copy link

todo bot commented Sep 11, 2019

UnmapProcessMemory: Verify that the src page list == dst page list.

In UnmapProcessMemory, we don't ensure that src_address is correct, that is, we don't check that the frames in the dst match the frame in the src. HOS/NX does this by building a PageList (essentially a vector of frames) and comparing them.
We could do something similar by iterating over the Mappings and checking if their Frames + PhysOffset are equals.


// TODO: UnmapProcessMemory: Verify that the src page list == dst page list.
// BODY: In UnmapProcessMemory, we don't ensure that src_address is correct,
// BODY: that is, we don't check that the frames in the dst match the frame
// BODY: in the src. HOS/NX does this by building a PageList (essentially
// BODY: a vector of frames) and comparing them.
// BODY:
// BODY: We could do something similar by iterating over the Mappings and
// BODY: checking if their Frames + PhysOffset are equals.
// Unmap.
while size != 0 {


This comment was generated by todo based on a TODO comment in e1d6e43 in #458. cc @roblabla.

@todo
Copy link

todo bot commented Sep 12, 2019

UnmapProcessMemory: Verify that the src page list == dst page list.

In UnmapProcessMemory, we don't ensure that src_address is correct, that is, we don't check that the frames in the dst match the frame in the src. HOS/NX does this by building a PageList (essentially a vector of frames) and comparing them.
We could do something similar by iterating over the Mappings and checking if their Frames + PhysOffset are equals.


// TODO: UnmapProcessMemory: Verify that the src page list == dst page list.
// BODY: In UnmapProcessMemory, we don't ensure that src_address is correct,
// BODY: that is, we don't check that the frames in the dst match the frame
// BODY: in the src. HOS/NX does this by building a PageList (essentially
// BODY: a vector of frames) and comparing them.
// BODY:
// BODY: We could do something similar by iterating over the Mappings and
// BODY: checking if their Frames + PhysOffset are equals.
// Unmap.
while size != 0 {


This comment was generated by todo based on a TODO comment in 2d7cdb3 in #458. cc @roblabla.

ProcessState::CreatedAttached => ProcessState::StartedAttached,
_ => unreachable!()
};
this.state.store(newstate, Ordering::SeqCst);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

textbook read-modify-write race condition.

You need to compare exchange the states, to arbitrate who's the winner of the race, and gets to actually start it.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is exactly why I believe we should be mutexing this stuff :P

- Process::new now takes a ProcInfo, and derives all necessary
information from this.
- Specify aslr base in a single location.
- Make UserLand address space start at 0x00200000
- Implement create_process syscall, as accurately as currently possible.
MapProcessMemory will allow loader to map the remote code created in
CreateProcess in its own address space in order to load the ELf into it.
This function mimicks nintendo's KMemoryManager::CheckRange. It goes
over a memory region and checks that it is in the expected state (both
in terms of MemoryState, MemoryPermissions and MemoryAttributes), and
that it is homogenous (that is, all submappings have the same state and
perms).

This is used to significantly simplify set_process_memory_permission and
map_process_memory.
Used to unmap memory mapped with MapProcessMemory. Loader now unmaps
the process memory it previously mapped.
Stop unwrapping left and right and return actual errors. This means we
should no longer be crashing if we send an invalid binary to loader.
Instead, loader will log an error and hum along.
Avoids starting a process twice. Will in the future allow tracking
crashes, and allow waiting on a process from userspace to see when it
changes state.
@todo
Copy link

todo bot commented Sep 17, 2019

UnmapProcessMemory: Verify that the src page list == dst page list.

In UnmapProcessMemory, we don't ensure that src_address is correct, that is, we don't check that the frames in the dst match the frame in the src. HOS/NX does this by building a PageList (essentially a vector of frames) and comparing them.
We could do something similar by iterating over the Mappings and checking if their Frames + PhysOffset are equals.


// TODO: UnmapProcessMemory: Verify that the src page list == dst page list.
// BODY: In UnmapProcessMemory, we don't ensure that src_address is correct,
// BODY: that is, we don't check that the frames in the dst match the frame
// BODY: in the src. HOS/NX does this by building a PageList (essentially
// BODY: a vector of frames) and comparing them.
// BODY:
// BODY: We could do something similar by iterating over the Mappings and
// BODY: checking if their Frames + PhysOffset are equals.
// Unmap.
while size != 0 {


This comment was generated by todo based on a TODO comment in 3b26b34 in #458. cc @roblabla.

@todo
Copy link

todo bot commented Sep 17, 2019

UnmapProcessMemory: Verify that the src page list == dst page list.

In UnmapProcessMemory, we don't ensure that src_address is correct, that is, we don't check that the frames in the dst match the frame in the src. HOS/NX does this by building a PageList (essentially a vector of frames) and comparing them.
We could do something similar by iterating over the Mappings and checking if their Frames + PhysOffset are equals.


// TODO: UnmapProcessMemory: Verify that the src page list == dst page list.
// BODY: In UnmapProcessMemory, we don't ensure that src_address is correct,
// BODY: that is, we don't check that the frames in the dst match the frame
// BODY: in the src. HOS/NX does this by building a PageList (essentially
// BODY: a vector of frames) and comparing them.
// BODY:
// BODY: We could do something similar by iterating over the Mappings and
// BODY: checking if their Frames + PhysOffset are equals.
// Unmap.
while size != 0 {


This comment was generated by todo based on a TODO comment in 06f20e2 in #458. cc @roblabla.

@todo
Copy link

todo bot commented Sep 17, 2019

UnmapProcessMemory: Verify that the src page list == dst page list.

In UnmapProcessMemory, we don't ensure that src_address is correct, that is, we don't check that the frames in the dst match the frame in the src. HOS/NX does this by building a PageList (essentially a vector of frames) and comparing them.
We could do something similar by iterating over the Mappings and checking if their Frames + PhysOffset are equals.


// TODO: UnmapProcessMemory: Verify that the src page list == dst page list.
// BODY: In UnmapProcessMemory, we don't ensure that src_address is correct,
// BODY: that is, we don't check that the frames in the dst match the frame
// BODY: in the src. HOS/NX does this by building a PageList (essentially
// BODY: a vector of frames) and comparing them.
// BODY:
// BODY: We could do something similar by iterating over the Mappings and
// BODY: checking if their Frames + PhysOffset are equals.
// Unmap.
while size != 0 {


This comment was generated by todo based on a TODO comment in 5b48fde in #458. cc @roblabla.

@todo
Copy link

todo bot commented Sep 18, 2019

UnmapProcessMemory: Verify that the src page list == dst page list.

In UnmapProcessMemory, we don't ensure that src_address is correct, that is, we don't check that the frames in the dst match the frame in the src. HOS/NX does this by building a PageList (essentially a vector of frames) and comparing them.
We could do something similar by iterating over the Mappings and checking if their Frames + PhysOffset are equals.


// TODO: UnmapProcessMemory: Verify that the src page list == dst page list.
// BODY: In UnmapProcessMemory, we don't ensure that src_address is correct,
// BODY: that is, we don't check that the frames in the dst match the frame
// BODY: in the src. HOS/NX does this by building a PageList (essentially
// BODY: a vector of frames) and comparing them.
// BODY:
// BODY: We could do something similar by iterating over the Mappings and
// BODY: checking if their Frames + PhysOffset are equals.
// Unmap.
while size != 0 {


This comment was generated by todo based on a TODO comment in 27ab3e4 in #458. cc @roblabla.

@roblabla
Copy link
Member Author

seriously, todobot, you're drunk.

Hopefully this time travis passes and we'll get this merged.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants