Is the Windows boot loader absent or is it just that the grub one is higher up in the order?

If the issue is just that you need to remove grub (but you didn’t delete the Windows boot loader), then you can do that without any installation media:

1. Open the command prompt as an Administator (e.g. press the Win key then search ‘command prompt’, right click and select ‘Run as administrator’)
2. Mount the EFI partition where the boot loaders are stored by running mountvol z: /s (you can use any letter if z is taken)
3. Move to this partition by running z:
4. Move to the folder with the boot loaders by running cd EFI
5. Remove the ubuntu boot loader by running rmdir /S ubuntu
6. Run dir to check what other boot loaders you have. There should a Microsoft one. Otherwise, you should look at other answers involving installation media.

Now when you power your computer on, it will use the Windows boot loader.