Merge branch 'master' of https://github.com/pmret/papermario into entity_funcs

This commit is contained in:
dark-samus 2021-01-16 15:12:49 -05:00
commit af694484f7
29 changed files with 2020 additions and 1620 deletions

3
.gitattributes vendored
View File

@ -1 +1,4 @@
*.h linguist-language=C
# suppress asm/nonmatchings/ in GitHub diffs
asm/nonmatchings/**/*.s linguist-generated

6
.gitignore vendored
View File

@ -9,13 +9,14 @@ venv/
.DS_Store
ctx.c
expected/
settings.mk
.vscode/launch.json
# Build artifacts
build.ninja
*.ld
*.z64
*.Yay0
*.msg.h
/build/
/docs/doxygen/
/include/ld_addrs.h
@ -33,4 +34,5 @@ settings.mk
/sprite/SpriteTable.xml
/mod.cfg
tools/Yay0compress
/tools/Yay0compress
/tools/n64crc

6
.vscode/tasks.json vendored
View File

@ -2,9 +2,9 @@
"version": "2.0.0",
"tasks": [
{
"label": "make",
"label": "ninja",
"type": "shell",
"command": "PM_HEADER_REBUILD=1 make all",
"command": "ninja",
"problemMatcher": [
{
"fileLocation": ["relative", "${workspaceFolder}"],
@ -35,7 +35,7 @@
{
"label": "diff",
"type": "shell",
"command": "PM_HEADER_REBUILD=1 ./diff.py -mwo ${input:funcName}",
"command": "./diff.py -mwo ${input:funcName}",
"isBackground": true,
"problemMatcher": [
{

View File

@ -1,20 +1,22 @@
## Contributing
# Contributing
### Dependencies
Thank you for your interest in contributing to this project!
## Dependencies
There are a few additional dependencies needed when contributing to this project. You can install them with `./install.sh --extra`.
### WSL
We provide [windows_terminal.bat](tools/windows_terminal.bat) to open a [Windows Terminal](https://www.microsoft.com/en-us/p/windows-terminal/9n0dx20hk701) with a recommended layout. Make sure a default distro (e.g. `wsl --set-default ubuntu`) is set beforehand.
## Build System
### Rebuilding
You will need to re-run `./configure.py` whenever `splat.yaml` or the number of source files changes (e.g. a function is matched and its `.s` file is removed). If you `git pull` and `ninja` breaks, you probably need to re-run `./configure.py`.
If you use Visual Studio Code, you can use _Run Build Task_ (Ctrl+Shift+B) to run `make`. Any errors or warnings generated by the compiler will show up in the _Problems_ tab.
If you use Visual Studio Code, you can use _Run Build Task_ (Ctrl+Shift+B) to run `ninja`. Any errors or warnings generated by the compiler will show up in the _Problems_ tab.
### Matching a function
#### Setup
## Tutorial: Matching a function
### Setup
Once you've created a successful (`OK`) build, copy `build/` to `expected/build/`:
@ -23,7 +25,7 @@ $ mkdir -p expected
$ cp -r build expected
```
#### Roughly converting assembly to C
### Roughly converting assembly to C
Decide on a function to match. These can be found in the subdirectories of `asm/nonmatchings/`. Currently, functions which use float constants, data sections, or jump tables are unmatchable.
@ -31,9 +33,7 @@ Take the relevant `.s` file and pass it to [mips_to_c](https://github.com/matt-k
You can also use mips_to_c locally installed to a destination of your choice. Then register a function in `~/.bashrc` that calls `path/to/mips_to_c.py (with args)`:
```
sudo apt update
sudo apt install python3-pip
sudo git clone https://github.com/matt-kempster/mips_to_c.git path/to/mips_to_c
git clone https://github.com/matt-kempster/mips_to_c /path/to/mips_to_c
```
Here's a starter function you can use:
@ -62,13 +62,13 @@ Fix any errors and rerun `diff.py`. This will involve typing the function signat
Once a successful build is made, `diff.py` will show you the difference between the original game's assembly (on the left) and what your C code generated (on the right).
#### Matching the function
### Matching the function
You're on your own now. Get your C code compiling to match the original assembly! `diff.py`, when running, will automatically recompile your code whenever you save the `.c` file.
If you use Visual Studio Code, you can use _Run Test Task_ to run `diff.py` and show you errors and warnings from the compiler inline. You might want to attach _Run Test Task_ to a keybinding, as you'll be using it often.
#### After matching
### After matching
Once you've matched a function, run the following scripts:
@ -78,3 +78,5 @@ $ ./format.sh
```
If `format.sh` has any problems with your code, go and fix the issues. If you can't fix a warning without making the function not match anymore, append `// NOLINT` to the offending line.
Then, please [create a pull request](https://github.com/pmret/papermario/pulls)!

62
INSTALL.md Normal file
View File

@ -0,0 +1,62 @@
# Setup
This repository supports:
- [Linux](#unix)
- [macOS](#unix)
- [Windows Subsystem for Linux 2](#wsl-2)
- [Docker](#docker) (any host OS)
If you encounter any issues setting up the repo, please feel free to [reach out to us on Discord](https://discord.gg/urUm3VG).
## Unix
Clone the repository:
```sh
git clone https://github.com/pmret/papermario
cd papermario
```
Install build dependencies:
```sh
./install.sh
```
Copy an unmodified Paper Mario (USA) ROM (sha1: `3837f44cda784b466c9a2d99df70d77c322b97a0`) into the root directory of the repository with the name `baserom.z64`. If you're using WSL, you can enter the Linux filesystem by opening `\\wsl$` in File Explorer; e.g. `\\wsl$\Ubuntu\home\<your username>\papermario`.
Configure the build and extract assets from the base ROM:
```sh
./configure.py
```
Compile the game:
```
ninja
```
If you get `papermario.z64: OK` at the end, the build succeeded!
The output ROM is `papermario.z64` - you can run this in any N64 emulator.
## WSL 2
1. Install or upgrade to **WSL 2** following [these instructions](https://aka.ms/wsl2-install) (Ubuntu is recommended)
2. Open a WSL terminal
3. Run the following command: `sudo apt update && sudo apt upgrade && cd ~`
4. Continue with [the instructions for Linux](#unix)
If you have Visual Studio Code, you can type `code .` to open the repo within it.
`Ctrl + J` opens up a Linux terminal within VS Code.
## Docker
A Docker image containing all dependencies can be built and ran as follows:
```sh
docker build . -t pm
docker run --rm -ti -v $(pwd):/papermario pm
```
Then continue with [the instructions for Linux](#unix).

7
Jenkinsfile vendored
View File

@ -4,14 +4,12 @@ pipeline {
stages {
stage('Setup') {
steps {
sh 'cp /usr/local/etc/roms/baserom_pm.z64 baserom.z64'
sh 'make setup'
sh './configure.py --baserom /usr/local/etc/roms/baserom_pm.z64'
}
}
stage('Build') {
steps {
echo 'Building...'
sh 'make -j'
sh 'ninja'
}
}
stage('Report Progress') {
@ -20,6 +18,7 @@ pipeline {
}
steps {
sh 'python3 progress.py --csv >> /var/www/papermar.io/html/reports/progress.csv'
sh 'python3 progress.py --shield-json > /var/www/papermar.io/html/reports/progress_shield.json'
}
}
}

314
Makefile
View File

@ -1,314 +0,0 @@
### Build Options ###
# Override these options in settings.mk or with `make SETTING=value'.
BASEROM = baserom.z64
TARGET = papermario
COMPARE = 1
NON_MATCHING = 0
WATCH_INCLUDES = 1
WSL_ELEVATE_GUI = 1
# Fail early if baserom does not exist
ifeq ($(wildcard $(BASEROM)),)
$(error Baserom `$(BASEROM)' not found.)
endif
# NON_MATCHING=1 implies COMPARE=0
ifeq ($(NON_MATCHING),1)
override COMPARE=0
endif
# PERMUTER=1 implies WATCH_INCLUDES=0
ifeq ($(PERMUTER),1)
override WATCH_INCLUDES=0
endif
### Output ###
BUILD_DIR := build
ROM := $(TARGET).z64
ELF := $(BUILD_DIR)/$(TARGET).elf
LD_SCRIPT := $(TARGET).ld
LD_MAP := $(BUILD_DIR)/$(TARGET).map
ASSETS_BIN := $(BUILD_DIR)/bin/assets/assets.bin
MSG_BIN := $(BUILD_DIR)/msg.bin
NPC_BIN := $(BUILD_DIR)/sprite/npc.bin
### Tools ###
PYTHON := python3
N64CKSUM := tools/n64crc
SPLAT_YAML := tools/splat.yaml
SPLAT = $(PYTHON) tools/n64splat/split.py $(BASEROM) $(SPLAT_YAML) .
YAY0COMPRESS = tools/Yay0compress
EMULATOR = mupen64plus
CROSS := mips-linux-gnu-
AS := $(CROSS)as
OLD_AS := tools/mips-nintendo-nu64-as
CC := tools/cc1
CPP := cpp
LD := $(CROSS)ld
OBJCOPY := $(CROSS)objcopy
WSL := 0
JAVA := java
UNAME_S := $(shell uname -s)
ifeq ($(UNAME_S),Linux)
OS=linux
ICONV := iconv --from UTF-8 --to SHIFT-JIS
ifeq ($(findstring microsoft,$(shell cat /proc/sys/kernel/osrelease)),microsoft)
WSL := 1
ifeq ($(WSL_ELEVATE_GUI),1)
JAVA := powershell.exe -command java
endif
endif
endif
ifeq ($(UNAME_S),Darwin)
OS=mac
ICONV := tools/iconv.py UTF-8 SHIFT-JIS
endif
OLD_AS=tools/$(OS)/mips-nintendo-nu64-as
CC=tools/$(OS)/cc1
### Compiler Options ###
CPPFLAGS := -Iinclude -Isrc -D _LANGUAGE_C -D _FINALROM -ffreestanding -DF3DEX_GBI_2 -D_MIPS_SZLONG=32 -Wundef -Wcomment
ASFLAGS := -EB -Iinclude -march=vr4300 -mtune=vr4300
OLDASFLAGS := -EB -Iinclude -G 0
CFLAGS := -O2 -quiet -G 0 -mcpu=vr4300 -mfix4300 -mips3 -mgp32 -mfp32 -Wimplicit -Wuninitialized -Wshadow
LDFLAGS := -T undefined_syms.txt -T undefined_syms_auto.txt -T undefined_funcs.txt -T undefined_funcs_auto.txt -T $(BUILD_DIR)/$(LD_SCRIPT) -Map $(LD_MAP) --no-check-sections
ifeq ($(WATCH_INCLUDES),1)
CPPMFLAGS = -MP -MD -MF $@.mk -MT $(BUILD_DIR)/$*.d
MDEPS = $(BUILD_DIR)/%.d
endif
ifeq ($(NON_MATCHING),1)
CPPFLAGS += -DNON_MATCHING
endif
-include settings.mk
### Sources ###
include sources.mk
ifeq ($(PERMUTER),1)
override OBJECTS:=$(filter %.c.o, $(OBJECTS))
endif
%.d: ;
ifeq ($(WATCH_INCLUDES),1)
-include $(foreach obj, $(OBJECTS), $(obj).mk)
endif
NPC_DIRS := $(foreach npc, $(NPC_SPRITES), sprite/npc/$(npc))
GENERATED_HEADERS := include/ld_addrs.h $(foreach dir, $(NPC_DIRS), include/$(dir).h)
### Targets ###
clean:
rm -rf $(BUILD_DIR) $(LD_SCRIPT)
clean-all:
rm -rf $(BUILD_DIR) bin msg img sprite .splat_cache $(LD_SCRIPT)
clean-code:
rm -rf $(BUILD_DIR)/src
tools:
make -C tools
setup: clean-all tools
@make split
split:
make $(LD_SCRIPT) -W $(SPLAT_YAML)
split-%:
$(SPLAT) --modes ld $* --verbose
split-all:
$(SPLAT) --modes all
test: $(ROM)
$(EMULATOR) $<
%.bin: $(LD_SCRIPT)
# Compressed files
%.Yay0: %
@mkdir -p $(shell dirname $@)
$(YAY0COMPRESS) $< $@
$(BUILD_DIR)/%.bin.Yay0: %.bin
@mkdir -p $(shell dirname $@)
$(YAY0COMPRESS) $< $@
# Data objects
$(BUILD_DIR)/%.bin.o: %.bin
@mkdir -p $(shell dirname $@)
$(LD) -r -b binary -o $@ $<
# Compressed data objects
$(BUILD_DIR)/%.Yay0.o: $(BUILD_DIR)/%.bin.Yay0
@mkdir -p $(shell dirname $@)
$(LD) -r -b binary -o $@ $<
# Compile C files
$(BUILD_DIR)/%.c.o: %.c $(MDEPS) | $(GENERATED_HEADERS)
@mkdir -p $(shell dirname $@)
$(CPP) $(CPPFLAGS) -o - $(CPPMFLAGS) $< | $(ICONV) | $(CC) $(CFLAGS) -o - | $(OLD_AS) $(OLDASFLAGS) -o $@ -
# Compile C files (with DSL macros)
$(foreach cfile, $(DSL_C_FILES), $(BUILD_DIR)/$(cfile).o): $(BUILD_DIR)/%.c.o: %.c $(MDEPS) tools/compile_dsl_macros.py | $(GENERATED_HEADERS)
@mkdir -p $(shell dirname $@)
$(CPP) $(CPPFLAGS) -o - $< $(CPPMFLAGS) | $(PYTHON) tools/compile_dsl_macros.py | $(ICONV) | $(CC) $(CFLAGS) -o - | $(OLD_AS) $(OLDASFLAGS) -o $@ -
# Assemble handwritten ASM
$(BUILD_DIR)/%.s.o: %.s
@mkdir -p $(shell dirname $@)
$(AS) $(ASFLAGS) -o $@ $<
# Data
$(BUILD_DIR)/data/%.data.o: asm/data/%.data.s
@mkdir -p $(shell dirname $@)
$(AS) $(ASFLAGS) -o $@ $<
# Rodata
$(BUILD_DIR)/rodata/%.rodata.o: asm/data/%.rodata.s
@mkdir -p $(shell dirname $@)
$(AS) $(ASFLAGS) -o $@ $<
# Images
$(BUILD_DIR)/%.png.o: $(BUILD_DIR)/%.png
$(LD) -r -b binary -o $@ $<
$(BUILD_DIR)/%.rgba16.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py rgba16 $< $@ $(IMG_FLAGS)
$(BUILD_DIR)/%.rgba32.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py rgba32 $< $@ $(IMG_FLAGS)
$(BUILD_DIR)/%.ci8.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py ci8 $< $@ $(IMG_FLAGS)
$(BUILD_DIR)/%.ci4.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py ci4 $< $@ $(IMG_FLAGS)
$(BUILD_DIR)/%.palette.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py palette $< $@ $(IMG_FLAGS)
$(BUILD_DIR)/%.ia4.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py ia4 $< $@ $(IMG_FLAGS)
$(BUILD_DIR)/%.ia8.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py ia8 $< $@ $(IMG_FLAGS)
$(BUILD_DIR)/%.ia16.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py ia16 $< $@ $(IMG_FLAGS)
$(BUILD_DIR)/%.i4.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py i4 $< $@ $(IMG_FLAGS)
$(BUILD_DIR)/%.i8.png: %.png
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/convert_image.py i8 $< $@ $(IMG_FLAGS)
# Assets
ASSET_FILES := $(foreach asset, $(ASSETS), $(BUILD_DIR)/bin/assets/$(asset))
YAY0_ASSET_FILES := $(foreach asset, $(filter-out %_tex, $(ASSET_FILES)), $(asset).Yay0)
$(BUILD_DIR)/bin/assets/%: bin/assets/%.bin
@mkdir -p $(shell dirname $@)
@cp $< $@
$(ASSETS_BIN): $(ASSET_FILES) $(YAY0_ASSET_FILES) sources.mk
@mkdir -p $(shell dirname $@)
@echo "building $@"
@$(PYTHON) tools/build_assets_bin.py $@ $(ASSET_FILES)
$(ASSETS_BIN:.bin=.o): $(ASSETS_BIN)
$(LD) -r -b binary -o $@ $<
# Messages
$(MSG_BIN): $(MESSAGES)
@mkdir -p $(shell dirname $@)
@echo "building $@"
@$(PYTHON) tools/compile_messages.py $@ /dev/null $(MESSAGES)
$(MSG_BIN:.bin=.o): $(MSG_BIN)
$(LD) -r -b binary -o $@ $<
# Sprites
$(foreach npc, $(NPC_SPRITES), $(eval $(BUILD_DIR)/sprite/npc/$(npc):: $(shell find sprite/npc/$(npc) -type f 2> /dev/null))) # dependencies
NPC_YAY0 := $(foreach npc, $(NPC_SPRITES), $(BUILD_DIR)/sprite/npc/$(npc).Yay0)
$(BUILD_DIR)/sprite/npc/%:: sprite/npc/% tools/compile_npc_sprite.py
@mkdir -p $(shell dirname $@)
$(PYTHON) tools/compile_npc_sprite.py $@ $<
$(NPC_BIN): $(NPC_YAY0) tools/compile_npc_sprites.py
@mkdir -p $(shell dirname $@)
@echo "building $@"
@$(PYTHON) tools/compile_npc_sprites.py $@ $(NPC_YAY0)
$(NPC_BIN:.bin=.o): $(NPC_BIN)
$(LD) -r -b binary -o $@ $<
include/sprite/npc/%.h: sprite/npc/%/SpriteSheet.xml tools/gen_sprite_animations_h.py
@mkdir -p $(shell dirname $@)
@echo "building $@"
@$(PYTHON) tools/gen_sprite_animations_h.py $@ sprite/npc/$* $(NPC_DIRS)
### Linker ###
$(LD_SCRIPT): $(SPLAT_YAML)
$(SPLAT) --modes ld bin Yay0 PaperMarioMapFS PaperMarioMessages img PaperMarioNpcSprites --new
$(BUILD_DIR)/$(LD_SCRIPT): $(LD_SCRIPT)
@mkdir -p $(shell dirname $@)
$(CPP) -P -DBUILD_DIR=$(BUILD_DIR) -o $@ $<
$(ROM): $(BUILD_DIR)/$(TARGET).bin
@cp $< $@
ifeq ($(COMPARE),1)
@sha1sum -c checksum.sha1 || (echo 'The build succeeded, but did not match the base ROM. This is expected if you are making changes to the game. To skip this check, use "make COMPARE=0".' && false)
endif
$(BUILD_DIR)/$(TARGET).elf: $(BUILD_DIR)/$(LD_SCRIPT) $(OBJECTS)
$(LD) $(LDFLAGS) -o $@
$(BUILD_DIR)/$(TARGET).bin: $(BUILD_DIR)/$(TARGET).elf
$(OBJCOPY) $< $@ -O binary
include/ld_addrs.h: $(BUILD_DIR)/$(LD_SCRIPT)
grep -E "[^\. ]+ =" $< -o | sed 's/^/extern void* /; s/ =/;/' > $@
### Star Rod (optional) ###
STAR_ROD := cd tools/star-rod && $(JAVA) -jar StarRod.jar
sprite/SpriteTable.xml: tools/star-rod sources.mk
$(PYTHON) tools/star-rod/spritetable.xml.py $(NPC_SPRITES) > $@
editor: tools/star-rod sprite/SpriteTable.xml
$(STAR_ROD)
### Make Settings ###
.PHONY: clean tools test setup split editor $(ROM)
.DELETE_ON_ERROR:
.SECONDARY:
.PRECIOUS: $(ROM) %.Yay0
.DEFAULT_GOAL := $(ROM)
# Remove built-in implicit rules to improve performance
MAKEFLAGS += --no-builtin-rules
# Fail targets if any command in the pipe exits with error
SHELL = /bin/bash -e -o pipefail

View File

@ -1,75 +1,22 @@
# Paper Mario
This is a WIP decompilation of Paper Mario (USA). It builds the following ROM:
[![Build Status][jenkins-badge]][jenkins] [![Progress][progress-badge]][progress] [![Discord Channel][discord-badge]][discord]
* papermario.z64 `md5: a722f8161ff489943191330bf8416496`
[jenkins]: https://jenkins.zelda64.dev/job/papermario/job/master
[jenkins-badge]: https://img.shields.io/jenkins/build?jobUrl=https%3A%2F%2Fjenkins.zelda64.dev%2Fjob%2Fpapermario%2Fjob%2Fmaster
Discord: [Paper Mario Modding](https://discord.gg/urUm3VG)
[progress]: https://papermar.io/progress
[progress-badge]: https://img.shields.io/endpoint?url=https://papermar.io/reports/progress_shield.json
## Setup
[discord]: https://discord.gg/urUm3VG
[discord-badge]: https://img.shields.io/discord/279322074412089344?color=%237289DA&logo=discord&logoColor=ffffff
You'll need Linux, a Linux VM, or [Windows 10 (WSL2)](#wsl) to work on this project.
This is a work-in-progress decompilation of Paper Mario (USA).
#### Clone the repository
```sh
$ git clone https://github.com/ethteck/papermario.git
$ cd papermario
```
It builds the following ROM:
#### Install build dependencies
* papermario.z64 `sha1: 3837f44cda784b466c9a2d99df70d77c322b97a0`
```sh
$ ./install.sh
```
To set up the repository, see [INSTALL.md](INSTALL.md).
Our install script does not yet support distros other than Ubuntu, Arch, and their derivatives. Please consider contributing to the script if you use another distro!
##### Docker
A Docker image containing all dependencies can be built and ran as follows:
```sh
# build image
$ docker build . -t pm
# spin up container, mounting current directory inside
$ docker run --rm -ti -v $(pwd):/papermario pm
```
#### Base ROM
You'll need a Paper Mario (USA) ROM to work on this project. Copy it into the root directory of the repository with the name `baserom.z64`.
#### Install tools and extract ROM
```sh
$ make setup
```
### Compile the game
```sh
$ make
```
If you get `OK`, you're all set! Otherwise, please feel free to reach out to us in [our Discord channel](https://discord.gg/urUm3VG).
## Star Rod GUI
You can open [our modified version of Star Rod](https://github.com/nanaian/star-rod-for-decomp) with `make editor`.
## Contributing
Please see [CONTRIBUTING.md](CONTRIBUTING.md) for information on how to contribute to the project. Any and all help is welcome!
## Troubleshooting
* If you are using Windows, you may encounter the following when you run `make`:
```
sha1sum -c checksum.sha1
sha1sum: 'papermario.z64'$'\r': No such file or directory
: FAILED open or read
sha1sum: WARNING: 1 listed file could not be read
Makefile:118: recipe for target 'verify' failed
make: *** [verify] Error 1
```
> 💡 Solution
> Run `git checkout checksum.sha1` and retry building. Windows has different line endings than Linux, causing some of our tools to break.

454
configure.py Executable file
View File

@ -0,0 +1,454 @@
#!/usr/bin/env python3
import re
import os, sys
from glob import glob
import ninja_syntax
from argparse import ArgumentParser
import asyncio
from subprocess import PIPE
import subprocess
import hashlib
sys.path.append(os.path.dirname(__file__) + "/tools/n64splat")
import split
INCLUDE_ASM_RE = re.compile(r"_INCLUDE_ASM\([^,]+, ([^,]+), ([^,)]+)") # note _ prefix
TARGET = "papermario"
NPC_SPRITES = "world_goombario world_kooper world_bombette world_parakarry world_bow world_watt world_sushie world_lakilester battle_goombario battle_kooper battle_bombette battle_parakarry battle_bow battle_watt battle_sushie battle_lakilester kooper_without_shell world_eldstar world_mamar world_skolar world_muskular world_misstar world_klevar world_kalmar battle_eldstar battle_mamar battle_skolar battle_muskular battle_misstar battle_klevar battle_kalmar twink jr_troopa spiked_jr_troopa spiked_para_jr_troopa mage_jr_troopa para_jr_troopa goomba spiked_goomba paragoomba koopa_troopa para_troopa fuzzy bob_omb bullet_bill bill_blaster monty_mole cleft pokey battle_bandit buzzy_beetle swooper stone_chomp putrid_piranha piranha_plant sentinel world_clubba battle_clubba shy_guy groove_guy sky_guy pyro_guy spy_guy medi_guy fuzzipede jungle_guy heart_plant hurt_plant m_bush bubble kent_c_koopa dayzee lakitu spiny bzzap ruff_puff spike_top duplighost albino_dino blooper baby_blooper gulpit dry_bones thrown_bone bony_beetle magikoopa flying_magikoopa world_koopatrol koopatrol hammer_bros bush_basic bush_blocky bush_dry bush_leafy bush_matted world_kammy battle_kammy goomba_bros goomba_king spiky_goomnut dark_toad koopa_bros buzzar tutankoopa chain_chomp world_tubba battle_tubba tubbas_heart big_lantern_ghost shy_squad_guy marshal_guy stilt_guy stilt_guy_unfold shy_stack_guy shy_stack_unfold shy_stack_damage shy_stack_rock general_guy general_guy_bomb tank_guy lava_piranha_head petit_piranha lava_bud huff_n_puff tuff_puff monstar crystal_king world_bowser battle_bowser luigi toad three_sisters vanna_t toad_kid toad_guard harry_t toad_minister postmaster conductor_toad train_station_toad fishmael artist_toad koopa koopa_without_shell world_bob_omb whacka dryite mouser boo yoshi yoshi_kid raven bubulb penguin shiver_toad world_bandit goompa goombaria gooma goompapa goomama the_master chan lee merlon chet_rippo rowf minh_t russ_t tayce_t fice_t bartender chanterelle rip_cheato chuck_quizmo merluvlee merlar merlow star_kid kolorado_wife koopa_koot kolorado battle_kolorado archeologist nomadimouse world_merlee battle_merlee disguised_moustafa moustafa oaklie bootler yakkey gourmet_guy village_leader leaders_friend rafael_raven tolielup gate_flower petunia posie lily rosie sun lakilulu ninji mayor_penguin mayor_penguin_wife penguin_patrol herringway merle star_rod fire coin parade_peach parade_koopas parade_burnt_bowser parade_luigi parade_partners parade_yoshis parade_kolorados parade_chicks parade_ice_show parade_toads parade_batons parade_drums parade_flags parade_horns parade_tubba_balloon parade_wizards parade_mario parade_shy_guys parade_twink leaf".split(" ")
MAPS = "dro_01 dro_02 hos_00 hos_01 hos_02 hos_03 hos_04 hos_05 hos_06 hos_10 hos_20 isk_01 isk_02 isk_03 isk_04 isk_05 isk_06 isk_07 isk_08 isk_09 isk_10 isk_11 isk_12 isk_13 isk_14 isk_16 isk_18 isk_19 iwa_00 iwa_01 iwa_02 iwa_03 iwa_04 iwa_10 iwa_11 osr_00 osr_01 osr_02 osr_03 kkj_00 kkj_01 kkj_02 kkj_03 kkj_10 kkj_11 kkj_12 kkj_13 kkj_14 kkj_15 kkj_16 kkj_17 kkj_18 kkj_19 kkj_20 kkj_21 kkj_22 kkj_23 kkj_24 kkj_25 kkj_26 kkj_27 kkj_28 kkj_29 kmr_00 kmr_02 kmr_03 kmr_04 kmr_05 kmr_06 kmr_07 kmr_09 kmr_10 kmr_11 kmr_12 kmr_20 kmr_30 kpa_01 kpa_03 kpa_04 kpa_08 kpa_09 kpa_10 kpa_11 kpa_12 kpa_13 kpa_14 kpa_15 kpa_16 kpa_17 kpa_32 kpa_33 kpa_40 kpa_41 kpa_50 kpa_52 kpa_60 kpa_61 kpa_62 kpa_63 kpa_70 kpa_80 kpa_90 kpa_91 kpa_94 kpa_95 kpa_96 kpa_102 kpa_111 kpa_112 kpa_113 kpa_115 kpa_116 kpa_117 kpa_118 kpa_119 kpa_121 kpa_130 kpa_133 kpa_134 machi mac_00 mac_01 mac_02 mac_03 mac_04 mac_05 mac_06 tik_01 tik_02 tik_03 tik_04 tik_05 tik_06 tik_07 tik_08 tik_09 tik_10 tik_12 tik_14 tik_15 tik_17 tik_18 tik_19 tik_20 tik_21 tik_22 tik_23 tik_25 kgr_01 kgr_02 nok_01 nok_02 nok_03 nok_04 nok_11 nok_12 nok_13 nok_14 nok_15 sbk_00 sbk_01 sbk_02 sbk_03 sbk_04 sbk_05 sbk_06 sbk_10 sbk_11 sbk_12 sbk_13 sbk_14 sbk_15 sbk_16 sbk_20 sbk_21 sbk_22 sbk_23 sbk_24 sbk_25 sbk_26 sbk_30 sbk_31 sbk_32 sbk_33 sbk_34 sbk_35 sbk_36 sbk_40 sbk_41 sbk_42 sbk_43 sbk_44 sbk_45 sbk_46 sbk_50 sbk_51 sbk_52 sbk_53 sbk_54 sbk_55 sbk_56 sbk_60 sbk_61 sbk_62 sbk_63 sbk_64 sbk_65 sbk_66 sbk_99 trd_00 trd_01 trd_02 trd_03 trd_04 trd_05 trd_06 trd_07 trd_08 trd_09 trd_10 tst_01 tst_02 tst_03 tst_04 tst_10 tst_11 tst_12 tst_13 tst_20 jan_00 jan_01 jan_02 jan_03 jan_04 jan_05 jan_06 jan_07 jan_08 jan_09 jan_10 jan_11 jan_12 jan_13 jan_14 jan_15 jan_16 jan_17 jan_18 jan_19 jan_22 jan_23 mim_01 mim_02 mim_03 mim_04 mim_05 mim_06 mim_07 mim_08 mim_09 mim_10 mim_11 mim_12 obk_01 obk_02 obk_03 obk_04 obk_05 obk_06 obk_07 obk_08 obk_09 arn_02 arn_03 arn_04 arn_05 arn_07 arn_08 arn_09 arn_10 arn_11 arn_12 arn_13 arn_20 dgb_01 dgb_02 dgb_03 dgb_04 dgb_05 dgb_06 dgb_07 dgb_08 dgb_09 dgb_10 dgb_11 dgb_12 dgb_13 dgb_14 dgb_15 dgb_16 dgb_17 dgb_18 kzn_01 kzn_02 kzn_03 kzn_04 kzn_05 kzn_06 kzn_07 kzn_08 kzn_09 kzn_10 kzn_11 kzn_17 kzn_18 kzn_19 kzn_20 kzn_22 kzn_23 flo_00 flo_03 flo_07 flo_08 flo_09 flo_10 flo_11 flo_12 flo_13 flo_14 flo_15 flo_16 flo_17 flo_18 flo_19 flo_21 flo_22 flo_23 flo_24 flo_25 sam_01 sam_02 sam_03 sam_04 sam_05 sam_06 sam_07 sam_08 sam_09 sam_10 sam_11 sam_12 pra_01 pra_02 pra_03 pra_04 pra_05 pra_09 pra_10 pra_11 pra_13 pra_14 pra_15 pra_16 pra_18 pra_19 pra_20 pra_21 pra_22 pra_29 pra_31 pra_32 pra_33 pra_34 pra_35 pra_40 omo_01 omo_02 omo_03 omo_04 omo_05 omo_06 omo_07 omo_08 omo_09 omo_10 omo_11 omo_12 omo_13 omo_14 omo_15 omo_16 omo_17 end_00 end_01 mgm_00 mgm_01 mgm_02 mgm_03 gv_01 kmr_bt03 kmr_bt04 kmr_bt05 kmr_bt06 nok_bt01 nok_bt02 nok_bt03 nok_bt04 trd_bt00 trd_bt01 trd_bt02 trd_bt03 trd_bt04 trd_bt05 iwa_bt01 iwa_bt02 sbk_bt02 isk_bt01 isk_bt02 isk_bt03 isk_bt04 isk_bt05 isk_bt06 isk_bt07 isk_bt08 arn_bt01 arn_bt02 arn_bt03 arn_bt04 arn_bt05 arn_bt06 dgb_bt01 dgb_bt02 dgb_bt03 dgb_bt04 dgb_bt05 mim_bt01 omo_bt01 omo_bt02 omo_bt03 omo_bt04 omo_bt05 omo_bt06 omo_bt07 kgr_bt01 flo_bt01 flo_bt02 flo_bt03 flo_bt04 flo_bt05 flo_bt06 jan_bt00 jan_bt01 jan_bt02 jan_bt03 jan_bt04 kzn_bt01 kzn_bt02 kzn_bt04 kzn_bt05 sam_bt01 sam_bt02 sam_bt03 sam_bt04 tik_bt01 tik_bt02 tik_bt03 tik_bt04 tik_bt05 pra_bt01 pra_bt02 pra_bt03 pra_bt04 mac_bt01 mac_bt02 kpa_bt01 kpa_bt02 kpa_bt03 kpa_bt04 kpa_bt05 kpa_bt07 kpa_bt08 kpa_bt09 kpa_bt11 kpa_bt13 kpa_bt14 hos_bt01 hos_bt02 kkj_bt01 kkj_bt02".split(" ")
ASSETS = sum([[f"{map_name}_shape", f"{map_name}_hit"] for map_name in MAPS], []) + "mac_tex tik_tex kgr_tex kmr_tex iwa_tex sbk_tex dro_tex isk_tex trd_tex nok_tex hos_tex kpa_tex osr_tex kkj_tex tst_tex jan_tex mim_tex obk_tex arn_tex dgb_tex kzn_tex flo_tex sam_tex pra_tex omo_tex end_tex mgm_tex gv__tex kmr_bg nok_bg sbk_bg sbk3_bg iwa_bg hos_bg arn_bg obk_bg omo_bg yos_bg jan_bg fla_bg flb_bg sra_bg yki_bg sam_bg kpa_bg title_bg title_data party_kurio party_kameki party_pinki party_pareta party_resa party_akari party_opuku party_pokopi".split(" ")
def obj(path: str):
if not path.startswith("$builddir/"):
path = "$builddir/" + path
return path + ".o"
def read_splat(splat_config: str):
import argparse
import yaml
# Load config
with open(splat_config) as f:
config = yaml.safe_load(f.read())
options = config.get("options")
assert options.get("ld_o_replace_extension", True) == False
# Initialize segments
all_segments = split.initialize_segments(options, splat_config, config["segments"])
objects = set()
segments = {}
for segment in all_segments:
for subdir, path, obj_type, start in segment.get_ld_files():
path = subdir + "/" + path
objects.add(path)
segments[path] = segment
# note: `objects` lacks .o extensions
return objects, segments
async def shell(cmd: str):
async with task_sem:
proc = await asyncio.create_subprocess_shell(cmd, stdout=PIPE, stderr=PIPE)
stdout, stderr = await proc.communicate()
assert proc.returncode == 0, f"{cmd} failed: {stderr}"
return stdout.decode("utf-8"), stderr.decode("utf-8")
async def task(coro):
global num_tasks, num_tasks_done
await coro
num_tasks_done += 1
print(f"\rConfiguring build... {(num_tasks_done / num_tasks) * 100:.0f}%", end="")
async def build_c_file(c_file: str, generated_headers, ccache, cppflags):
# preprocess c_file, but simply put an _ in front of INCLUDE_ASM and SCRIPT
stdout, stderr = await shell(f"{ccache} {cpp} {cppflags} '-DINCLUDE_ASM(...)=_INCLUDE_ASM(__VA_ARGS__)' '-DSCRIPT(...)=_SCRIPT(__VA_ARGS__)' {c_file} -o -")
# search for macro usage (note _ prefix)
uses_dsl = "_SCRIPT(" in stdout
s_deps = []
for line in stdout.splitlines():
if line.startswith("_INCLUDE_ASM"):
match = INCLUDE_ASM_RE.match(line)
if match:
s_deps.append("asm/nonmatchings/" + eval(match[1]) + "/" + match[2] + ".s")
# add build task to ninja
n.build(obj(c_file), "cc_dsl" if uses_dsl else "cc", c_file, implicit=s_deps, order_only=generated_headers)
def build_yay0_file(bin_file: str):
yay0_file = f"$builddir/{os.path.splitext(bin_file)[0]}.Yay0"
n.build(yay0_file, "yay0compress", bin_file, implicit="tools/Yay0compress")
build_bin_object(yay0_file)
def build_bin_object(bin_file: str):
n.build(obj(bin_file), "bin", bin_file)
def build_image(f: str, segment):
path, img_type, png = f.rsplit(".", 2)
out = "$builddir/" + path + "." + img_type + ".png"
flags = ""
if img_type != "palette":
if segment.flip_horizontal:
flags += "--flip-x"
if segment.flip_vertical:
flags += "--flip-y"
n.build(out, "img", path + ".png", implicit="tools/convert_image.py", variables={
"img_type": img_type,
"img_flags": flags,
})
build_bin_object(out)
def cmd_exists(cmd):
return subprocess.call("type " + cmd, shell=True,
stdout=subprocess.PIPE, stderr=subprocess.PIPE) == 0
async def main():
global n, cpp, task_sem, num_tasks, num_tasks_done
task_sem = asyncio.Semaphore(8)
parser = ArgumentParser(description="Paper Mario build.ninja generator")
parser.add_argument("--cpp", help="GNU C preprocessor command")
parser.add_argument("--baserom", default="baserom.z64", help="Path to unmodified Paper Mario (U) z64 ROM")
parser.add_argument("--cflags", default="", help="Extra cc/cpp flags")
args = parser.parse_args()
# on macOS, /usr/bin/cpp defaults to clang rather than gcc (but we need gcc's)
if args.cpp is None and sys.platform == "darwin" and "Free Software Foundation" not in (await shell("cpp --version"))[0]:
print("error: system C preprocessor is not GNU!")
print("This is a known issue on macOS - only clang's cpp is installed by default.")
print("Use 'brew' to obtain GNU cpp, then run this script again with the --cpp option, e.g.")
print(" ./configure.py --cpp cpp-10")
exit(1)
# verify baserom exists and is clean
try:
with open(args.baserom, "rb") as f:
h = hashlib.sha1()
h.update(f.read())
if h.hexdigest() != "3837f44cda784b466c9a2d99df70d77c322b97a0":
print(f"error: baserom '{args.baserom}' is modified, refusing to split it!")
print("The baserom must be an unmodified Paper Mario (U) z64 ROM.")
exit(1)
except IOError:
print(f"error: baserom '{args.baserom}' does not exist!")
print(f"Please make sure an unmodified Paper Mario (U) z64 ROM exists at '{args.baserom}'.")
if args.baserom == "baserom.z64": # the default
print("Or run this script again with the --baserom option:")
print(" ./configure.py --baserom /path/to/papermario.z64")
exit(1)
cpp = args.cpp or "cpp"
ccache = "ccache" if cmd_exists("ccache") else ""
# compile n64splat dependencies
await shell("make -C tools/n64splat")
# split assets
print("Splitting segments from baserom", end="")
split.main(
args.baserom,
"tools/splat.yaml",
".",
[ "ld", "bin", "Yay0", "PaperMarioMapFS", "PaperMarioMessages", "img", "PaperMarioNpcSprites" ],
False,
False,
)
print("")
print("Configuring build...", end="")
# generate build.ninja
n = ninja_syntax.Writer(open("build.ninja", "w"), width=120)
cppflags = "-Iinclude -Isrc -D _LANGUAGE_C -D _FINALROM -ffreestanding -DF3DEX_GBI_2 -D_MIPS_SZLONG=32 " + args.cflags
n.variable("builddir", "build")
n.variable("target", TARGET)
n.variable("cross", "mips-linux-gnu-")
n.variable("python", sys.executable)
if sys.platform == "darwin":
os_dir = "mac"
elif sys.platform == "linux":
if os.uname()[4] == "aarch64":
os_dir = "arm"
else:
os_dir = "linux"
else:
print(f"Unsupported platform {sys.platform}")
sys.exit(1)
n.variable("os", os_dir)
n.variable("iconv", "tools/iconv.py UTF-8 SHIFT-JIS" if sys.platform == "darwin" else "iconv --from UTF-8 --to SHIFT-JIS")
n.variable("cppflags", f"{cppflags} -Wcomment")
n.variable("cflags", "-O2 -quiet -G 0 -mcpu=vr4300 -mfix4300 -mips3 -mgp32 -mfp32 -Wuninitialized -Wshadow " + args.cflags)
n.newline()
n.rule("cc",
command=f"bash -o pipefail -c '{ccache} {cpp} $cppflags $in -o - | $iconv | {ccache} tools/$os/cc1 $cflags -o - | tools/$os/mips-nintendo-nu64-as -EB -G 0 - -o $out'",
description="cc $in",
depfile="$out.d",
deps="gcc")
n.rule("cc_dsl",
command=f"bash -o pipefail -c '{ccache} {cpp} $cppflags $in -o - | $python tools/compile_dsl_macros.py | $iconv | {ccache} tools/$os/cc1 $cflags -o - | tools/$os/mips-nintendo-nu64-as -EB -G 0 - -o $out'",
description="cc (with dsl) $in",
depfile="$out.d",
deps="gcc")
n.newline()
n.rule("cpp",
command=f"{cpp} -P -DBUILD_DIR=$builddir $in -o $out",
description="cc (with dsl) $in",
depfile="$out.d",
deps="gcc")
n.newline()
n.rule("yay0compress",
command=f"tools/Yay0compress $in $out",
description="compress $in")
n.newline()
n.rule("bin",
command="${cross}ld -r -b binary $in -o $out",
description="bin $in")
n.newline()
n.rule("as",
command="${cross}as -EB -march=vr4300 -mtune=vr4300 -Iinclude $in -o $out",
description="assemble $in")
n.newline()
# $img_type, $img_flags
n.rule("img",
command="$python tools/convert_image.py $img_type $in $out $img_flags",
description="image $in")
n.newline()
# $sprite_id, $sprite_dir, $sprite_name
n.rule("sprite_animations_h",
command="$python tools/gen_sprite_animations_h.py $out $sprite_dir $sprite_id",
description="sprite_animations_h $sprite_name ($sprite_id)")
n.rule("npc_sprite",
command="$python tools/compile_npc_sprite.py $out $sprite_dir",
description="npc_sprite $sprite_name ($sprite_id)")
n.rule("npc_sprites",
command="$python tools/compile_npc_sprites.py $out $in",
description="package npc sprites")
n.newline()
n.rule("ld_addrs_h",
command="grep -E \"[^\. ]+ =\" $in -o | sed 's/^/extern void* /; s/ =/;/' > $out",
description="ld_addrs_h $in")
n.newline()
# $msg_combine_headers
n.rule("msg_combine",
command="$python tools/msg/combine.py $out $in --headers $msg_combine_headers",
description="combine messages")
n.rule("msg",
command="$python tools/msg/parse_compile.py $in $out",
description="msg $in")
n.newline()
n.rule("assets",
command="$python tools/build_assets_bin.py $out $in",
description="combine assets")
n.newline()
n.rule("link",
command="${cross}ld -T undefined_syms.txt -T undefined_syms_auto.txt -T undefined_funcs.txt -T undefined_funcs_auto.txt -Map $builddir/$target.map --no-check-sections -T $in -o $out",
description="link $out")
n.newline()
n.rule("rom",
command="${cross}objcopy $in $out -O binary && tools/n64crc $out",
description="rom $in")
n.newline()
n.rule("sha1sum",
command="sha1sum -c $in && touch $out",
description="compare")
n.newline()
n.rule("cc_modern_exe", command="cc $in -O3 -o $out")
n.newline()
objects, segments = read_splat("tools/splat.yaml") # no .o extensions!
c_files = (f for f in objects if f.endswith(".c")) # glob("src/**/*.c", recursive=True)
n.comment("target")
n.build("$builddir/$target.ld", "cpp", "$target.ld")
n.build("$builddir/$target.elf", "link", "$builddir/$target.ld", implicit=[obj(o) for o in objects], implicit_outputs="$builddir/$target.map")
n.build("$target.z64", "rom", "$builddir/$target.elf", implicit="tools/n64crc")
n.build("$builddir/is_ok", "sha1sum", "checksum.sha1", implicit="$target.z64")
n.newline()
n.default("$builddir/is_ok")
n.newline()
# generated headers
n.comment("generated headers")
generated_headers = []
def add_generated_header(h: str):
generated_headers.append(h)
if not os.path.exists(h):
# mkdir -p
os.makedirs(os.path.dirname(h), exist_ok=True)
# touch it so cpp doesn't complain if its #included
open(h, "w").close()
# mark it as really old so ninja builds it
os.utime(h, (0, 0))
return h
n.build(add_generated_header("include/ld_addrs.h"), "ld_addrs_h", "$builddir/$target.ld")
# messages
msg_files = glob("src/**/*.msg", recursive=True) + glob("msg/**/*.msg", recursive=True)
for msg_file in msg_files:
n.build(
f"$builddir/{msg_file}.bin",
"msg",
msg_file,
implicit="tools/msg/parse_compile.py",
)
n.build(
"$builddir/msg.bin",
"msg_combine",
[f"$builddir/{msg_file}.bin" for msg_file in msg_files],
implicit="tools/msg/combine.py",
implicit_outputs=[add_generated_header(f"{msg_file}.h") for msg_file in msg_files],
variables={ "msg_combine_headers": [f"{msg_file}.h" for msg_file in msg_files] }
)
n.build("$builddir/msg.o", "bin", "$builddir/msg.bin")
# sprites
npc_sprite_yay0s = []
for sprite_id, sprite_name in enumerate(NPC_SPRITES, 1):
sources = glob(f"sprite/npc/{sprite_name}/**/*.*", recursive=True)
variables = {
"sprite_name": sprite_name,
"sprite_dir": f"sprite/npc/{sprite_name}",
"sprite_id": sprite_id,
}
# generated header
n.build(
add_generated_header(f"include/sprite/npc/{sprite_name}.h"),
"sprite_animations_h",
implicit=sources + ["tools/gen_sprite_animations_h.py"],
variables=variables,
)
# sprite bin/yay0
n.build(
f"$builddir/sprite/npc/{sprite_name}",
"npc_sprite",
implicit=sources + ["tools/compile_npc_sprite.py"],
variables=variables,
)
yay0 = f"$builddir/sprite/npc/{sprite_name}.Yay0"
npc_sprite_yay0s.append(yay0)
n.build(
yay0,
"yay0compress",
f"$builddir/sprite/npc/{sprite_name}",
implicit=["tools/Yay0compress"],
)
n.newline()
# fast tasks
n.comment("data")
for f in objects:
segment = segments[f]
if f.endswith(".c"):
continue # these are handled later
elif f.endswith(".Yay0"):
build_yay0_file(os.path.splitext(f)[0] + ".bin")
elif f.endswith(".bin"):
build_bin_object(f)
elif f.endswith(".data"):
n.build(obj(f), "as", "asm/" + f + ".s")
elif f.endswith(".rodata"):
n.build(obj(f), "as", "asm/" + f[2:] + ".s")
elif f.endswith(".s"):
n.build(obj(f), "as", f)
elif f.endswith(".png"):
build_image(f, segment)
elif f == "sprite/npc":
# combine sprites
n.build(f"$builddir/{f}.bin", "npc_sprites", npc_sprite_yay0s, implicit="tools/compile_npc_sprites.py")
n.build(obj(f), "bin", f"$builddir/{f}.bin")
elif f == "/msg": # XXX: why does this have a leading '/'??
continue # done already above
elif f == "bin/assets/assets":
asset_files = [] # even indexes: uncompressed; odd indexes: compressed
for asset_name in ASSETS:
if asset_name.endswith("_tex"): # uncompressed
asset_files.append(f"bin/assets/{asset_name}.bin")
asset_files.append(f"bin/assets/{asset_name}.bin")
else: # uncompressed
source_file = f"bin/assets/{asset_name}.bin"
asset_file = f"$builddir/assets/{asset_name}.Yay0"
asset_files.append(source_file)
asset_files.append(asset_file)
n.build(asset_file, "yay0compress", source_file, implicit="tools/Yay0compress")
n.build("$builddir/assets.bin", "assets", asset_files)
n.build(obj(f), "bin", "$builddir/assets.bin")
else:
print("warning: dont know what to do with object " + f)
n.newline()
n.build("generated_headers", "phony", generated_headers)
n.newline()
# slow tasks generated concurrently
n.comment("c")
tasks = [task(build_c_file(f, "generated_headers", ccache, cppflags)) for f in c_files]
num_tasks = len(tasks)
num_tasks_done = 0
await asyncio.gather(*tasks)
print("")
n.newline()
# c tools that need to be compiled
n.build("tools/Yay0compress", "cc_modern_exe", "tools/Yay0compress.c")
n.build("tools/n64crc", "cc_modern_exe", "tools/n64crc.c")
n.newline()
print("")
print("Build configuration complete! Now run")
print(" ninja")
print(f"to compile '{TARGET}.z64'.")
if __name__ == "__main__":
asyncio.run(main())

836
diff.py

File diff suppressed because it is too large Load Diff

View File

@ -5,4 +5,4 @@ def apply(config, args):
config['myimg'] = 'papermario.z64'
config['mapfile'] = 'build/papermario.map'
config['source_directories'] = ['.']
config['makeflags'] = ['COMPARE=0', 'WATCH_INCLUDES=0']
config['make_command'] = ['ninja']

View File

@ -4,8 +4,10 @@
#include "common.h"
#ifndef SPLAT
#ifndef INCLUDE_ASM
#define INCLUDE_ASM(TYPE, FOLDER, NAME, ARGS...) \
TYPE __attribute__((naked)) NAME(ARGS) { __asm__( ".include \"include/macro.inc\"\n.include \"asm/nonmatchings/"FOLDER"/"#NAME".s\"\n.set reorder\n.set at"); }
#endif
#else
#define INCLUDE_ASM(TYPE, FOLDER, NAME, ARGS...)
#endif

View File

@ -4,7 +4,7 @@
if cat /etc/os-release | grep ID=ubuntu &> /dev/null; then
echo "Installing packages for Ubuntu (apt)"
sudo apt install -y git python3 python3-pip python3-setuptools build-essential binutils-mips-linux-gnu zlib1g-dev libyaml-dev || exit 1
sudo apt install -y git python3 python3-pip python3-setuptools build-essential binutils-mips-linux-gnu zlib1g-dev libyaml-dev ninja-build || exit 1
python3 -m pip install -U -r requirements.txt
if [[ $1 == "--extra" ]]; then
@ -25,7 +25,7 @@ if cat /etc/os-release | grep ID=arch &> /dev/null; then
sudo pacman -Syu || exit 1
# Install dependencies
sudo pacman -S --noconfirm --needed git python python-pip python-setuptools base-devel zlib libyaml || exit 1
sudo pacman -S --noconfirm --needed git python python-pip python-setuptools base-devel zlib libyaml ninja || exit 1
python3 -m pip install -U -r requirements.txt
# Install binutils if required
@ -62,7 +62,7 @@ fi
if cat /etc/os-release | grep ID=opensuse &> /dev/null; then
echo "Installing packages for openSUSE (zypper)"
sudo zypper -n install git python3 python3-devel python3-pip python3-setuptools gcc gcc-c++ glibc-devel make cross-mips-binutils zlib-devel libyaml-devel
sudo zypper -n install git python3 python3-devel python3-pip python3-setuptools gcc gcc-c++ glibc-devel make cross-mips-binutils zlib-devel libyaml-devel ninja
# Link the openSUSE locations for binutils tools to their usual GNU locations
sudo ln -s /usr/bin/mips-suse-linux-addr2line /usr/bin/mips-linux-gnu-addr2line
@ -103,7 +103,7 @@ if cat /etc/os-release | grep ID=alpine &> /dev/null; then
fi
# Install dependencies
sudo apk add --no-cache bash wget git python3 python3-dev py3-pip build-base zlib-dev yaml-dev
sudo apk add --no-cache bash wget git python3 python3-dev py3-pip build-base zlib-dev yaml-dev ninja
python3 -m pip install -U -r requirements.txt
# Install binutils if required

View File

@ -79,6 +79,16 @@ def main(args):
csv_list = [str(version), timestamp, git_hash, str(len(all_funcs)), str(len(nonmatching_funcs)),
str(len(matching_funcs)), str(total_size), str(nonmatching_size), str(matching_size)]
print(",".join(csv_list))
elif args.shield_json:
import json
# https://shields.io/endpoint
print(json.dumps({
"schemaVersion": 1,
"label": "progress",
"message": f"{matching_ratio:.2f}%",
"color": "yellow",
}))
else:
if matching_size + nonmatching_size != total_size:
print("Warning: category/total size mismatch!\n")
@ -89,6 +99,7 @@ def main(args):
if __name__ == "__main__":
parser = argparse.ArgumentParser(description="Reports progress for the project")
parser.add_argument("--csv", action="store_true")
parser.add_argument("--shield-json", action="store_true")
args = parser.parse_args()
main(args)

View File

@ -4,3 +4,5 @@ lark-parser
python-ranges
pypng
colorama
ninja_syntax
msgpack

View File

@ -3,3 +3,4 @@ cxxfilt
python-Levenshtein
stringcase
watchdog
gitpython

View File

@ -1,64 +0,0 @@
OBJECTS := $(foreach OBJECT, $(shell $(PYTHON) tools/n64splat/list_objects.py $(SPLAT_YAML)), $(BUILD_DIR)/$(OBJECT))
DSL_C_FILES := $(shell grep -lrF "SCRIPT" src)
MAPS := \
dro_01 dro_02 \
hos_00 hos_01 hos_02 hos_03 hos_04 hos_05 hos_06 hos_10 hos_20 \
isk_01 isk_02 isk_03 isk_04 isk_05 isk_06 isk_07 isk_08 isk_09 isk_10 isk_11 isk_12 isk_13 isk_14 isk_16 isk_18 isk_19 \
iwa_00 iwa_01 iwa_02 iwa_03 iwa_04 iwa_10 iwa_11 \
osr_00 osr_01 osr_02 osr_03 kkj_00 kkj_01 kkj_02 kkj_03 kkj_10 kkj_11 kkj_12 kkj_13 kkj_14 kkj_15 kkj_16 kkj_17 kkj_18 kkj_19 kkj_20 kkj_21 kkj_22 kkj_23 kkj_24 kkj_25 kkj_26 kkj_27 kkj_28 kkj_29 \
kmr_00 kmr_02 kmr_03 kmr_04 kmr_05 kmr_06 kmr_07 kmr_09 kmr_10 kmr_11 kmr_12 kmr_20 kmr_30 \
kpa_01 kpa_03 kpa_04 kpa_08 kpa_09 kpa_10 kpa_11 kpa_12 kpa_13 kpa_14 kpa_15 kpa_16 kpa_17 kpa_32 kpa_33 kpa_40 kpa_41 kpa_50 kpa_52 kpa_60 kpa_61 kpa_62 kpa_63 kpa_70 kpa_80 kpa_90 kpa_91 kpa_94 kpa_95 kpa_96 kpa_102 kpa_111 kpa_112 kpa_113 kpa_115 kpa_116 kpa_117 kpa_118 kpa_119 kpa_121 kpa_130 kpa_133 kpa_134 \
machi mac_00 mac_01 mac_02 mac_03 mac_04 mac_05 mac_06 \
tik_01 tik_02 tik_03 tik_04 tik_05 tik_06 tik_07 tik_08 tik_09 tik_10 tik_12 tik_14 tik_15 tik_17 tik_18 tik_19 tik_20 tik_21 tik_22 tik_23 tik_25 \
kgr_01 kgr_02 \
nok_01 nok_02 nok_03 nok_04 nok_11 nok_12 nok_13 nok_14 nok_15 \
sbk_00 sbk_01 sbk_02 sbk_03 sbk_04 sbk_05 sbk_06 sbk_10 sbk_11 sbk_12 sbk_13 sbk_14 sbk_15 sbk_16 sbk_20 sbk_21 sbk_22 sbk_23 sbk_24 sbk_25 sbk_26 sbk_30 sbk_31 sbk_32 sbk_33 sbk_34 sbk_35 sbk_36 sbk_40 sbk_41 sbk_42 sbk_43 sbk_44 sbk_45 sbk_46 sbk_50 sbk_51 sbk_52 sbk_53 sbk_54 sbk_55 sbk_56 sbk_60 sbk_61 sbk_62 sbk_63 sbk_64 sbk_65 sbk_66 sbk_99 \
trd_00 trd_01 trd_02 trd_03 trd_04 trd_05 trd_06 trd_07 trd_08 trd_09 trd_10 \
tst_01 tst_02 tst_03 tst_04 tst_10 tst_11 tst_12 tst_13 tst_20 \
jan_00 jan_01 jan_02 jan_03 jan_04 jan_05 jan_06 jan_07 jan_08 jan_09 jan_10 jan_11 jan_12 jan_13 jan_14 jan_15 jan_16 jan_17 jan_18 jan_19 jan_22 jan_23 \
mim_01 mim_02 mim_03 mim_04 mim_05 mim_06 mim_07 mim_08 mim_09 mim_10 mim_11 mim_12 \
obk_01 obk_02 obk_03 obk_04 obk_05 obk_06 obk_07 obk_08 obk_09 \
arn_02 arn_03 arn_04 arn_05 arn_07 arn_08 arn_09 arn_10 arn_11 arn_12 arn_13 arn_20 \
dgb_01 dgb_02 dgb_03 dgb_04 dgb_05 dgb_06 dgb_07 dgb_08 dgb_09 dgb_10 dgb_11 dgb_12 dgb_13 dgb_14 dgb_15 dgb_16 dgb_17 dgb_18 \
kzn_01 kzn_02 kzn_03 kzn_04 kzn_05 kzn_06 kzn_07 kzn_08 kzn_09 kzn_10 kzn_11 kzn_17 kzn_18 kzn_19 kzn_20 kzn_22 kzn_23 \
flo_00 flo_03 flo_07 flo_08 flo_09 flo_10 flo_11 flo_12 flo_13 flo_14 flo_15 flo_16 flo_17 flo_18 flo_19 flo_21 flo_22 flo_23 flo_24 flo_25 \
sam_01 sam_02 sam_03 sam_04 sam_05 sam_06 sam_07 sam_08 sam_09 sam_10 sam_11 sam_12 \
pra_01 pra_02 pra_03 pra_04 pra_05 pra_09 pra_10 pra_11 pra_13 pra_14 pra_15 pra_16 pra_18 pra_19 pra_20 pra_21 pra_22 pra_29 pra_31 pra_32 pra_33 pra_34 pra_35 pra_40 \
omo_01 omo_02 omo_03 omo_04 omo_05 omo_06 omo_07 omo_08 omo_09 omo_10 omo_11 omo_12 omo_13 omo_14 omo_15 omo_16 omo_17 \
end_00 end_01 \
mgm_00 mgm_01 mgm_02 mgm_03 \
gv_01 \
kmr_bt03 kmr_bt04 kmr_bt05 kmr_bt06 \
nok_bt01 nok_bt02 nok_bt03 nok_bt04 \
trd_bt00 trd_bt01 trd_bt02 trd_bt03 trd_bt04 trd_bt05 \
iwa_bt01 iwa_bt02 \
sbk_bt02 \
isk_bt01 isk_bt02 isk_bt03 isk_bt04 isk_bt05 isk_bt06 isk_bt07 isk_bt08 \
arn_bt01 arn_bt02 arn_bt03 arn_bt04 arn_bt05 arn_bt06 \
dgb_bt01 dgb_bt02 dgb_bt03 dgb_bt04 dgb_bt05 \
mim_bt01 \
omo_bt01 omo_bt02 omo_bt03 omo_bt04 omo_bt05 omo_bt06 omo_bt07 \
kgr_bt01 flo_bt01 flo_bt02 flo_bt03 flo_bt04 flo_bt05 flo_bt06 \
jan_bt00 jan_bt01 jan_bt02 jan_bt03 jan_bt04 \
kzn_bt01 kzn_bt02 kzn_bt04 kzn_bt05 sam_bt01 sam_bt02 sam_bt03 sam_bt04 \
tik_bt01 tik_bt02 tik_bt03 tik_bt04 tik_bt05 \
pra_bt01 pra_bt02 pra_bt03 pra_bt04 mac_bt01 mac_bt02 \
kpa_bt01 kpa_bt02 kpa_bt03 kpa_bt04 kpa_bt05 kpa_bt07 kpa_bt08 kpa_bt09 kpa_bt11 kpa_bt13 kpa_bt14 \
hos_bt01 hos_bt02 \
kkj_bt01 kkj_bt02
ASSETS := \
$(foreach map, $(MAPS), $(map)_shape $(map)_hit) \
mac_tex tik_tex kgr_tex kmr_tex iwa_tex sbk_tex dro_tex isk_tex trd_tex nok_tex hos_tex kpa_tex osr_tex kkj_tex tst_tex jan_tex mim_tex obk_tex arn_tex dgb_tex kzn_tex flo_tex sam_tex pra_tex omo_tex end_tex mgm_tex gv__tex \
kmr_bg nok_bg sbk_bg sbk3_bg iwa_bg hos_bg arn_bg obk_bg omo_bg yos_bg jan_bg fla_bg flb_bg sra_bg yki_bg sam_bg kpa_bg title_bg \
title_data \
party_kurio party_kameki party_pinki party_pareta party_resa party_akari party_opuku party_pokopi
MESSAGES := $(shell find msg -type f -name "*.msg" 2> /dev/null)
NPC_SPRITES := world_goombario world_kooper world_bombette world_parakarry world_bow world_watt world_sushie world_lakilester battle_goombario battle_kooper battle_bombette battle_parakarry battle_bow battle_watt battle_sushie battle_lakilester kooper_without_shell world_eldstar world_mamar world_skolar world_muskular world_misstar world_klevar world_kalmar battle_eldstar battle_mamar battle_skolar battle_muskular battle_misstar battle_klevar battle_kalmar twink jr_troopa spiked_jr_troopa spiked_para_jr_troopa mage_jr_troopa para_jr_troopa goomba spiked_goomba paragoomba koopa_troopa para_troopa fuzzy bob_omb bullet_bill bill_blaster monty_mole cleft pokey battle_bandit buzzy_beetle swooper stone_chomp putrid_piranha piranha_plant sentinel world_clubba battle_clubba shy_guy groove_guy sky_guy pyro_guy spy_guy medi_guy fuzzipede jungle_guy heart_plant hurt_plant m_bush bubble kent_c_koopa dayzee lakitu spiny bzzap ruff_puff spike_top duplighost albino_dino blooper baby_blooper gulpit dry_bones thrown_bone bony_beetle magikoopa flying_magikoopa world_koopatrol koopatrol hammer_bros bush_basic bush_blocky bush_dry bush_leafy bush_matted world_kammy battle_kammy goomba_bros goomba_king spiky_goomnut dark_toad koopa_bros buzzar tutankoopa chain_chomp world_tubba battle_tubba tubbas_heart big_lantern_ghost shy_squad_guy marshal_guy stilt_guy stilt_guy_unfold shy_stack_guy shy_stack_unfold shy_stack_damage shy_stack_rock general_guy general_guy_bomb tank_guy lava_piranha_head petit_piranha lava_bud huff_n_puff tuff_puff monstar crystal_king world_bowser battle_bowser luigi toad three_sisters vanna_t toad_kid toad_guard harry_t toad_minister postmaster conductor_toad train_station_toad fishmael artist_toad koopa koopa_without_shell world_bob_omb whacka dryite mouser boo yoshi yoshi_kid raven bubulb penguin shiver_toad world_bandit goompa goombaria gooma goompapa goomama the_master chan lee merlon chet_rippo rowf minh_t russ_t tayce_t fice_t bartender chanterelle rip_cheato chuck_quizmo merluvlee merlar merlow star_kid kolorado_wife koopa_koot kolorado battle_kolorado archeologist nomadimouse world_merlee battle_merlee disguised_moustafa moustafa oaklie bootler yakkey gourmet_guy village_leader leaders_friend rafael_raven tolielup gate_flower petunia posie lily rosie sun lakilulu ninji mayor_penguin mayor_penguin_wife penguin_patrol herringway merle star_rod fire coin parade_peach parade_koopas parade_burnt_bowser parade_luigi parade_partners parade_yoshis parade_kolorados parade_chicks parade_ice_show parade_toads parade_batons parade_drums parade_flags parade_horns parade_tubba_balloon parade_wizards parade_mario parade_shy_guys parade_twink leaf
# Image settings
$(BUILD_DIR)/img/battle/text_action_command_ratings.ia4.png: IMG_FLAGS = --flip-y

View File

@ -3,7 +3,3 @@
#define NAMESPACE b_area_kmr_part_2
#include "world/common/SomeMatrixOperations.inc.c"
// INCLUDE_ASM(s32, "battle/area_kmr_part_2/43A5A0", func_80218B10_43A5A0);
// INCLUDE_ASM(s32, "battle/area_kmr_part_2/43A5A0", func_80218B80_43A610);

View File

@ -1,11 +0,0 @@
default: all
all: Yay0compress
make -C n64splat
Yay0compress:
gcc Yay0compress.c -O3 -o Yay0compress
clean:
rm -f Yay0compress
make clean -C n64splat

BIN
tools/arm/cc1 Executable file

Binary file not shown.

BIN
tools/arm/mips-nintendo-nu64-as Executable file

Binary file not shown.

View File

@ -3,6 +3,7 @@
import os
from sys import argv
from pathlib import Path
from itertools import tee
def next_multiple(pos, multiple):
return pos + pos % multiple
@ -10,7 +11,7 @@ def next_multiple(pos, multiple):
def build_mapfs(out_bin, assets):
# every TOC entry's name field has data after the null terminator made up from all the previous name fields.
# we probably don't have to do this for the game to read the data properly (it doesn't read past the null terminator
# of `string`), but the original devs' equivalent to build_assets_fs.py had this bug so we need to replicate it to match.
# of `string`), but the original devs' equivalent of this script had this bug so we need to replicate it to match.
written_names = []
with open(out_bin, "wb") as f:
@ -19,17 +20,9 @@ def build_mapfs(out_bin, assets):
next_data_pos = (len(assets) + 1) * 0x1C
asset_idx = 0
for decompressed in assets:
for decompressed, compressed in assets:
toc_entry_pos = 0x20 + asset_idx * 0x1C
decompressed = Path(decompressed)
compressed = decompressed.with_suffix(".Yay0")
# non-texture assets should be compressed
if not decompressed.stem.endswith("_tex") and not compressed.exists():
print(f"uncompressed asset: {decompressed} (expected {compressed} to exist)")
exit(1)
# data for TOC entry
name = decompressed.stem + "\0"
offset = next_data_pos
@ -72,4 +65,10 @@ if __name__ == "__main__":
argv.pop(0) # python3
out = argv.pop(0)
build_mapfs(out, argv)
assets = []
# pairs
for i in range(0, len(argv), 2):
assets.append((Path(argv[i]), Path(argv[i+1])))
build_mapfs(out, assets)

View File

@ -1,823 +0,0 @@
#! /usr/bin/python3
from sys import argv
from collections import OrderedDict
import re
class Message:
def __init__(self, name, section, index):
self.name = name
self.section = section
self.index = index
self.bytes = []
def try_convert_int(s):
try:
return int(s, base=0)
except:
return s
def parse_command(source):
if source[0] != "[":
return None, [], {}, source
source = source[1:] # "["
inside_brackets = ""
while source[0] != "]":
if source[0] == "\n":
return None, [], {}, source
inside_brackets += source[0]
source = source[1:]
source = source[1:] # "]"
command, *args = inside_brackets.split(" ")
positional_args = []
named_args = {}
if "=" in command:
key, value = command.split("=", 1)
command = key
named_args[key] = try_convert_int(value)
for arg in args:
if "=" in arg:
key, value = arg.split("=", 1)
named_args[key.lower()] = try_convert_int(value.lower())
else:
positional_args.append(try_convert_int(arg))
return command.lower(), positional_args, named_args, source
def color_to_code(color, ctx="normal"):
COLORS = {
"normal": {
"normal": 0x0A,
"red": 0x20,
"pink": 0x21,
"purple": 0x22,
"blue": 0x23,
"cyan": 0x24,
"green": 0x25,
"yellow": 0x26,
},
"diary": {
"normal": 0x00,
"red": 0x07,
},
"inspect": {
"dark": 0x17,
},
"button": {
"blue": 0x10,
"green": 0x11,
"red": 0x12,
"yellow": 0x13,
"gray": 0x14,
"grey": 0x14,
},
"popup": {
"red": 0x28,
"pink": 0x29,
"purple": 0x2A,
"blue": 0x2B,
"teal": 0x2C,
"green": 0x2D,
"yellow": 0x2E,
"normal": 0x2F,
},
"sign": {
"normal": 0x18,
"red": 0x19,
"blue": 0x1A,
"green": 0x1B,
}
}
if type(color) is int:
return color
return COLORS.get(ctx, {}).get(color)
CHARSET = {
"𝅘𝅥𝅮": 0x00,
"!": 0x01,
'"': 0x02,
"#": 0x03,
"$": 0x04,
"%": 0x05,
"&": 0x06,
"'": 0x07,
"(": 0x08,
")": 0x09,
"*": 0x0A,
"+": 0x0B,
",": 0x0C,
"-": 0x0D,
".": 0x0E,
"/": 0x0F,
"0": 0x10,
"1": 0x11,
"2": 0x12,
"3": 0x13,
"4": 0x14,
"5": 0x15,
"6": 0x16,
"7": 0x17,
"8": 0x18,
"9": 0x19,
":": 0x1A,
";": 0x1B,
"<": 0x1C,
"=": 0x1D,
">": 0x1E,
"?": 0x1F,
"@": 0x20,
"A": 0x21,
"B": 0x22,
"C": 0x23,
"D": 0x24,
"E": 0x25,
"F": 0x26,
"G": 0x27,
"H": 0x28,
"I": 0x29,
"J": 0x2A,
"K": 0x2B,
"L": 0x2C,
"M": 0x2D,
"N": 0x2E,
"O": 0x2F,
"P": 0x30,
"Q": 0x31,
"R": 0x32,
"S": 0x33,
"T": 0x34,
"U": 0x35,
"V": 0x36,
"W": 0x37,
"X": 0x38,
"Y": 0x39,
"Z": 0x3A,
"[": 0x3B,
"¥": 0x3C,
"]": 0x3D,
"^": 0x3E,
"_": 0x3F,
"`": 0x40,
"a": 0x41,
"b": 0x42,
"c": 0x43,
"d": 0x44,
"e": 0x45,
"f": 0x46,
"g": 0x47,
"h": 0x48,
"i": 0x49,
"j": 0x4A,
"k": 0x4B,
"l": 0x4C,
"m": 0x4D,
"n": 0x4E,
"o": 0x4F,
"p": 0x50,
"q": 0x51,
"r": 0x52,
"s": 0x53,
"t": 0x54,
"u": 0x55,
"v": 0x56,
"w": 0x57,
"x": 0x58,
"y": 0x59,
"z": 0x5A,
"{": 0x5B,
"|": 0x5C,
"}": 0x5D,
"~": 0x5E,
"°": 0x5F,
"À": 0x60,
"Á": 0x61,
"Â": 0x62,
"Ä": 0x63,
"Ç": 0x64,
"È": 0x65,
"É": 0x66,
"Ê": 0x67,
"Ë": 0x68,
"Ì": 0x69,
"Í": 0x6A,
"Î": 0x6B,
"Ï": 0x6C,
"Ñ": 0x6D,
"Ò": 0x6E,
"Ó": 0x6F,
"Ô": 0x70,
"Ö": 0x71,
"Ù": 0x72,
"Ú": 0x73,
"Û": 0x74,
"Ü": 0x75,
"ß": 0x76,
"à": 0x77,
"á": 0x78,
"â": 0x79,
"ä": 0x7A,
"ç": 0x7B,
"è": 0x7C,
"é": 0x7D,
"ê": 0x7E,
"ë": 0x7F,
"ì": 0x80,
"í": 0x81,
"î": 0x82,
"ï": 0x83,
"ñ": 0x84,
"ò": 0x85,
"ó": 0x86,
"ô": 0x87,
"ö": 0x88,
"ù": 0x89,
"ú": 0x8A,
"û": 0x8B,
"ü": 0x8C,
"¡": 0x8D,
"¿": 0x8E,
"ª": 0x8F,
"": 0x90,
"": 0x91,
"": 0x92,
"": 0x93,
"": 0x94,
"": 0x95,
"": 0x96,
"": 0x97,
"": 0xA2,
"": 0xA3,
"": 0xA4,
"": 0xA5,
" ": 0xF7,
"": [0xFF, 0x24, 0xFF, 0x05, 0x10, 0x98, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x11, 0x99, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x12, 0xA1, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x13, 0x9D, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x13, 0x9E, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x13, 0x9F, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x13, 0xA0, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x14, 0x9A, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x14, 0x9B, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x14, 0x9C, 0xFF, 0x25],
}
CHARSET_CREDITS = {
"A": 0x00,
"B": 0x01,
"C": 0x02,
"D": 0x03,
"E": 0x04,
"F": 0x05,
"G": 0x06,
"H": 0x07,
"I": 0x08,
"J": 0x09,
"K": 0x0A,
"L": 0x0B,
"M": 0x0C,
"N": 0x0D,
"O": 0x0E,
"P": 0x0F,
"Q": 0x10,
"R": 0x11,
"S": 0x12,
"T": 0x13,
"U": 0x14,
"V": 0x15,
"W": 0x16,
"X": 0x17,
"Y": 0x18,
"Z": 0x19,
"'": 0x1A,
".": 0x1B,
",": 0x1C,
"0": 0x1D,
"1": 0x1E,
"2": 0x1F,
"3": 0x20,
"4": 0x21,
"5": 0x22,
"6": 0x23,
"7": 0x24,
"8": 0x25,
"9": 0x26,
"©": 0x27,
"&": 0x28,
" ": 0xF7,
}
if __name__ == "__main__":
if len(argv) < 4:
print("usage: compile_messages.py [OUTBIN] [OUTHEADER] [INFILES]")
exit(1)
_, outfile, outheader, *infiles = argv
messages = []
for filename in infiles:
message = None
with open(filename, "r") as f:
source = f.read()
lineno = 1
charset = CHARSET
font_stack = [0]
sound_stack = [0]
color_stack = [0x0A]
while len(source) > 0:
if source.startswith("\n"):
lineno += 1
source = source[1:]
continue
if message is None:
if source.startswith("//"):
while source[0] != "\n":
source = source[1:]
else:
command, positional_args, named_args, source = parse_command(source)
if not command:
print(f"{filename}:{lineno}: expected [message]")
exit(1)
name = positional_args[0] if len(positional_args) > 0 else None
message = Message(name, named_args.get("section"), named_args.get("index"))
messages.append(message)
else:
command, positional_args, named_args, source = parse_command(source)
if command:
if command == "/message":
message.bytes += [0xFD]
# padding
while len(message.bytes) % 4 != 0:
message.bytes += [0x00]
message = None
elif command == "raw":
message.bytes += [*positional_args]
elif command == "func":
message.bytes += [0xFF, *positional_args]
elif command == "br":
message.bytes += [0xF0]
elif command == "prompt":
message.bytes += [0xF1]
elif command == "sleep":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: {command} command requires a positional parameter")
exit(1)
message.bytes += [0xF2, positional_args[0]]
elif command == "next":
message.bytes += [0xFB]
elif command == "color":
if "color" not in named_args:
print(f"{filename}:{lineno}: color command requires a 'color' parameter")
exit(1)
color = color_to_code(**named_args)
if color is None:
print(f"{filename}:{lineno}: unknown color combination {named_args}")
exit(1)
message.bytes += [0xFF, 0x05, color]
color_stack.append(color)
elif command == "/color":
color_stack.pop()
message.bytes += [0xFF, 0x05, color_stack[0]]
elif command == "style":
if "style" not in named_args:
print(f"{filename}:{lineno}: style command requires a 'style' parameter")
exit(1)
message.bytes += [0xFC]
style = named_args["style"]
if type(style) is int:
message.bytes += [style, *positional_args]
else:
if style == "right":
message.bytes += [0x01]
elif style == "left":
message.bytes += [0x02]
elif style == "center":
message.bytes += [0x03]
elif style == "tattle":
message.bytes += [0x04]
elif style == "choice":
if "w" not in named_args or "h" not in named_args or "x" not in named_args or "y" not in named_args:
print(f"{filename}:{lineno}: 'choice' style requires parameters: x, y, w, h")
exit(1)
message.bytes += [0x05, named_args["x"], named_args["y"], named_args["w"], named_args["h"]]
elif style == "inspect":
message.bytes += [0x06]
elif style == "sign":
message.bytes += [0x07]
elif style == "lamppost":
message.bytes += [0x08]
elif style == "postcard":
message.bytes += [0x09]
elif style == "popup":
message.bytes += [0x0A]
elif style == "upgrade":
if "w" not in named_args or "h" not in named_args or "x" not in named_args or "y" not in named_args:
print(f"{filename}:{lineno}: 'upgrade' style requires parameters: x, y, w, h")
exit(1)
message.bytes += [0x0C, named_args["w"], named_args["x"], named_args["h"], named_args["y"]]
elif style == "narrate":
message.bytes += [0x0D]
elif style == "epilogue":
message.bytes += [0x0E]
elif command == "font":
if "font" not in named_args:
print(f"{filename}:{lineno}: font command requires a 'font' parameter")
exit(1)
font = named_args["font"]
if font == "normal":
font = 0
elif font == "title":
font = 3
elif font == "subtitle":
font = 4
if type(font) is not int:
print(f"{filename}:{lineno}: unknown font '{font}'")
exit(1)
message.bytes += [0xFF, 0x00, font]
font_stack.append(font)
if font == 3 or font == 4:
charset = CHARSET_CREDITS
else:
charset = CHARSET
elif command == "/font":
font_stack.pop()
message.bytes += [0xFF, 0x00, font_stack[0]]
if font == 3 or font == 4:
charset = CHARSET_CREDITS
else:
charset = CHARSET
elif command == "noskip":
message.bytes += [0xFF, 0x07]
elif command == "/noskip":
message.bytes += [0xFF, 0x08]
elif command == "instant":
message.bytes += [0xFF, 0x09]
elif command == "/instant":
message.bytes += [0xFF, 0x0A]
elif command == "kerning":
if "kerning" not in named_args:
print(f"{filename}:{lineno}: kerning command requires a 'kerning' parameter")
exit(1)
message.bytes += [0xFF, 0x0B, named_args["kerning"]]
elif command == "scroll":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: scroll command requires a positional parameter")
exit(1)
message.bytes += [0xFF, 0x0C, positional_args[0]]
elif command == "size":
if "x" not in named_args or "y" not in named_args:
print(f"{filename}:{lineno}: size command requires parameters: x, y")
exit(1)
message.bytes += [0xFF, 0x0D, named_args["x"], named_args["y"]]
elif command == "/size":
message.bytes += [0xFF, 0x0E]
elif command == "speed":
if "delay" not in named_args or "chars" not in named_args:
print(f"{filename}:{lineno}: speed command requires parameters: delay, chars")
exit(1)
message.bytes += [0xFF, 0x0F, named_args["delay"], named_args["chars"]]
elif command == "pos":
if "y" not in named_args:
print(f"{filename}:{lineno}: pos command requires parameter: y (x is optional)")
exit(1)
if "x" in named_args:
message.bytes += [0xFF, 0x10, named_args["x"], named_args["y"]]
else:
message.bytes += [0xFF, 0x11, named_args["y"]]
elif command == "indent":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: indent command requires a positional parameter")
exit(1)
message.bytes += [0xFF, 0x12, positional_args[0]]
elif command == "down":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: down command requires a positional parameter")
exit(1)
message.bytes += [0xFF, 0x13, positional_args[0]]
elif command == "up":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: up command requires a positional parameter")
exit(1)
message.bytes += [0xFF, 0x14, positional_args[0]]
elif command == "image":
if len(positional_args) == 1:
message.bytes += [0xFF, 0x15, positional_args[0]]
elif len(positional_args) == 7:
message.bytes += [0xFF, 0x18, *positional_args]
else:
print(f"{filename}:{lineno}: image command requires 1 or 7 positional parameters")
exit(1)
elif command == "sprite":
if len(positional_args) != 3:
print(f"{filename}:{lineno}: sprite command requires 3 positional parameters")
exit(1)
message.bytes += [0xFF, 0x16, *positional_args]
elif command == "item":
if len(positional_args) != 2:
print(f"{filename}:{lineno}: item command requires 2 positional parameters")
exit(1)
message.bytes += [0xFF, 0x17, *positional_args]
elif command == "cursor":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: cursor command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x1E, *positional_args]
elif command == "option":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: option command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x21, *positional_args]
elif command == "choice":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: choice command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x1E, positional_args[0], 0xFF, 0x21, positional_args[0]]
elif command == "choicecount":
if "choicecount" not in named_args:
print(f"{filename}:{lineno}: choicecount command requires a 'choicecount' parameter")
exit(1)
message.bytes += [0xFF, 0x1F, named_args["choicecount"]]
elif command == "cancel":
if "cancel" not in named_args:
print(f"{filename}:{lineno}: cancel command requires a 'cancel' parameter")
exit(1)
message.bytes += [0xFF, 0x20, named_args["cancel"]]
elif command == "shaky":
message.bytes += [0xFF, 0x26, 0x00]
elif command == "/shaky":
message.bytes += [0xFF, 0x27, 0x00]
elif command == "wavy":
message.bytes += [0xFF, 0x26, 0x01]
elif command == "/wavy":
message.bytes += [0xFF, 0x27, 0x01]
elif command == "shaky":
if "opacity" in named_args:
print(f"{filename}:{lineno}: shaky command doesn't accept parameter 'fade' (hint: did you mean 'faded-shaky'?)")
exit(1)
message.bytes += [0xFF, 0x26, 0x00]
elif command == "/shaky":
message.bytes += [0xFF, 0x27, 0x00]
elif command == "noise":
message.bytes += [0xFF, 0x26, 0x03, named_args.get("fade", 3)]
elif command == "/noise":
message.bytes += [0xFF, 0x27, 0x03]
elif command == "faded-shaky":
message.bytes += [0xFF, 0x26, 0x05, named_args.get("fade", 5)]
elif command == "/faded-shaky":
message.bytes += [0xFF, 0x27, 0x05]
elif command == "fade":
message.bytes += [0xFF, 0x26, 0x07, named_args.get("fade", 7)]
elif command == "/fade":
message.bytes += [0xFF, 0x27, 0x07]
elif command == "shout" or command == "shrinking":
message.bytes += [0xFF, 0x26, 0x0A]
elif command == "/shout" or command == "/shrinking":
message.bytes += [0xFF, 0x27, 0x0A]
elif command == "whisper" or command == "growing":
message.bytes += [0xFF, 0x26, 0x0B]
elif command == "/whisper" or command == "/growing":
message.bytes += [0xFF, 0x27, 0x0B]
elif command == "scream" or command == "shaky-size":
message.bytes += [0xFF, 0x26, 0x0C]
elif command == "/scream" or command == "/shaky-size":
message.bytes += [0xFF, 0x27, 0x0C]
elif command == "chortle" or command == "wavy-size":
message.bytes += [0xFF, 0x26, 0x0D]
elif command == "/chortle" or command == "/wavy-size":
message.bytes += [0xFF, 0x27, 0x0D]
elif command == "shadow":
message.bytes += [0xFF, 0x26, 0x0E]
elif command == "/shadow":
message.bytes += [0xFF, 0x27, 0x0E]
elif command == "var":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: var command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x28, *positional_args]
elif command == "center":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: center command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x29, *positional_args]
elif command == "volume":
if "volume" not in named_args:
print(f"{filename}:{lineno}: volume command requires a 'volume' parameter")
exit(1)
message.bytes += [0xFF, 0x2E, named_args["volume"]]
elif command == "sound":
if "sound" not in named_args:
print(f"{filename}:{lineno}: sound command requires a 'sound' parameter")
exit(1)
sound = named_args["sound"]
if sound == "normal":
sound = 0
elif sound == "bowser":
sound = 1
elif sound == "spirit":
sound = 2
if type(sound) is not int:
print(f"{filename}:{lineno}: unknown sound '{sound}'")
exit(1)
message.bytes += [0xFF, 0x2F, sound]
sound_stack.append(sound)
elif command == "/sound":
sound_stack.pop()
message.bytes += [0xFF, 0x2F, sound_stack[0]]
elif command == "a":
color_code = color_to_code(named_args.get("color", "blue"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x98, 0xFF, 0x25]
elif command == "b":
color_code = color_to_code(named_args.get("color", "green"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x99, 0xFF, 0x25]
elif command == "l":
color_code = color_to_code(named_args.get("color", "gray"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9A, 0xFF, 0x25]
elif command == "r":
color_code = color_to_code(named_args.get("color", "gray"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9B, 0xFF, 0x25]
elif command == "z":
color_code = color_to_code(named_args.get("color", "gray"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9C, 0xFF, 0x25]
elif command == "c-up":
color_code = color_to_code(named_args.get("color", "yellow"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9D, 0xFF, 0x25]
elif command == "c-down":
color_code = color_to_code(named_args.get("color", "yellow"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9E, 0xFF, 0x25]
elif command == "c-left":
color_code = color_to_code(named_args.get("color", "yellow"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9F, 0xFF, 0x25]
elif command == "c-right":
color_code = color_to_code(named_args.get("color", "yellow"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0xA0, 0xFF, 0x25]
elif command == "start":
color_code = color_to_code(named_args.get("color", "red"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0xA1, 0xFF, 0x25]
elif command == "note":
message.bytes += [0x00]
elif command == "heart":
message.bytes += [0x90]
elif command == "star":
message.bytes += [0x91]
elif command == "arrow-up":
message.bytes += [0x92]
elif command == "arrow-down":
message.bytes += [0x93]
elif command == "arrow-left":
message.bytes += [0x94]
elif command == "arrow-right":
message.bytes += [0x95]
elif command == "circle":
message.bytes += [0x96]
elif command == "cross":
message.bytes += [0x97]
elif command == "wait":
print(f"{filename}:{lineno}: unknown command 'wait' (hint: did you mean 'prompt'?)")
exit(1)
elif command == "pause":
print(f"{filename}:{lineno}: unknown command 'pause' (hint: did you mean 'sleep'?)")
exit(1)
else:
print(f"{filename}:{lineno}: unknown command '{command}'")
exit(1)
else:
if source[0] == "\\":
source = source[1:]
if source[0] in charset:
data = charset[source[0]]
if type(data) is int:
message.bytes.append(data)
else:
message.bytes += data
source = source[1:]
else:
print(f"{filename}:{lineno}: unsupported character '{source[0]}' for current font")
exit(1)
if message != None:
print(f"{filename}: missing [/message]")
exit(1)
with open(outfile, "wb") as f:
messages.sort(key=lambda msg: bool(msg.section) + bool(msg.index))
names = OrderedDict()
sections = [] * 0x2E
for message in messages:
if message.section is None:
# allocate a section
for section_idx, section in enumerate(sections):
if len(section) < 0xFFF:
break
else:
section_idx = message.section
while len(sections) <= section_idx:
sections.append([])
section = sections[section_idx]
index = message.index if message.index is not None else len(section)
if message.name:
if message.name in names:
print(f"warning: multiple messages with name '{message.name}'")
names[message.name] = (section_idx, index)
section.append(bytes(message.bytes))
f.seek((len(sections) + 1) * 4) # skip past table of contents
section_offsets = []
for section in sections:
message_offsets = []
for message in section:
message_offsets.append(f.tell())
f.write(message)
section_offset = f.tell()
section_offsets.append(section_offset)
for offset in message_offsets:
f.write(offset.to_bytes(4, byteorder="big"))
f.write(section_offset.to_bytes(4, byteorder="big"))
# padding
while f.tell() % 0x10 != 0:
f.write(b'\0\0\0\0')
f.seek(0)
for offset in section_offsets:
f.write(offset.to_bytes(4, byteorder="big"))
f.write(b'\0\0\0\0')
with open(outheader, "w") as f:
f.write(
"#ifndef _MESSAGE_IDS_H_\n"
"#define _MESSAGE_IDS_H_\n"
"\n"
'#include "messages.h"\n'
"\n"
)
for name, i in names.items():
section, index = i
f.write(f"#define MessageID_{name} MESSAGE_ID({section}, {index})\n")
f.write("\n#endif\n")

View File

@ -8,14 +8,14 @@ from splat_ext.PaperMarioNpcSprites import Sprite
if __name__ == "__main__":
if len(argv) < 4:
print("usage: gen_sprite_animations_h.py [OUT] [DIR] [ALLDIRS]")
print("usage: gen_sprite_animations_h.py [OUT] [DIR] [ID]")
exit(1)
_, outfile, sprite_dir, *alldirs = argv
_, outfile, sprite_dir, s = argv
with open(outfile, "w") as f:
# get sprite index
s = alldirs.index(sprite_dir) + 1
s = int(s)
assert s >= 1
sprite_dir = Path(sprite_dir)

125
tools/msg/combine.py Executable file
View File

@ -0,0 +1,125 @@
#! /usr/bin/python3
from sys import argv
from collections import OrderedDict
import re
import msgpack
import os
class Message:
def __init__(self, d: dict, header_file_index: int):
self.section = d.get("section")
self.index = d.get("index")
self.name = d.get("name")
self.bytes = d["bytes"]
self.header_file_index = header_file_index
if __name__ == "__main__":
if len(argv) < 3:
print("usage: combine.py [out.bin] [compiled...] --headers [out.h]")
exit(1)
_, outfile, *infiles = argv
messages = []
header_files = []
for i, infile in enumerate(infiles):
if infile == "--headers":
header_files = infiles[i+1:]
break
with open(infile, "rb") as f:
messages.extend(Message(msg, i) for msg in msgpack.unpack(f))
with open(outfile, "wb") as f:
# sectioned+indexed, followed by just sectioned, followed by just indexed, followed by named (unsectioned & unindexed)
messages.sort(key=lambda msg: bool(msg.section)<<2 + bool(msg.index))
names = set()
sections = [] * 0x2E
messages_by_file = {}
for message in messages:
if message.section is None:
# allocate a section
for section_idx, section in enumerate(sections):
if len(section) < 0xFFF:
break
else:
section_idx = message.section
while len(sections) <= section_idx:
sections.append([])
section = sections[section_idx]
index = message.index if message.index is not None else len(section)
if message.name:
if message.name in names:
print(f"warning: multiple messages with name '{message.name}'")
else:
names.add(message.name)
if message.header_file_index in messages_by_file:
messages_by_file[message.header_file_index].add(message)
else:
messages_by_file[message.header_file_index] = set([message])
section.append(message.bytes)
f.seek((len(sections) + 1) * 4) # skip past table of contents
section_offsets = []
for section in sections:
message_offsets = []
for message in section:
message_offsets.append(f.tell())
f.write(message)
section_offset = f.tell()
section_offsets.append(section_offset)
for offset in message_offsets:
f.write(offset.to_bytes(4, byteorder="big"))
f.write(section_offset.to_bytes(4, byteorder="big"))
# padding
while f.tell() % 0x10 != 0:
f.write(b'\0\0\0\0')
f.seek(0)
for offset in section_offsets:
f.write(offset.to_bytes(4, byteorder="big"))
f.write(b'\0\0\0\0')
for i, header_file in enumerate(header_files):
messages = messages_by_file.get(i, [])
h = (
f"#ifndef _MESSAGE_IDS_{i}_H_\n"
f"#define _MESSAGE_IDS_{i}_H_\n"
"\n"
'#include "messages.h"\n'
"\n"
)
for message in messages:
h += f"#define MessageID_{message.name} MESSAGE_ID({message.section}, {message.index})\n"
h += "\n#endif\n"
h_lines = h.splitlines()
# this doesnt work properly with ninja. the build is fast enough anyway
"""
# only rewrite the header file if its content changed
with open(header_file, "r") as f:
cur_h_lines = f.read().splitlines()
is_different = cur_h_lines != h_lines
if is_different:
with open(header_file, "w") as f:
f.write(h)
"""
with open(header_file, "w") as f:
f.write(h)

764
tools/msg/parse_compile.py Normal file
View File

@ -0,0 +1,764 @@
#! /usr/bin/python3
from sys import argv
from collections import OrderedDict
import re
import msgpack # way faster than pickle
class Message:
def __init__(self, name, section, index):
self.name = name
self.section = section
self.index = index
self.bytes = [] # XXX: bytearray would be better
def try_convert_int(s):
try:
return int(s, base=0)
except:
return s
def parse_command(source):
if source[0] != "[":
return None, [], {}, source
source = source[1:] # "["
inside_brackets = ""
while source[0] != "]":
if source[0] == "\n":
return None, [], {}, source
inside_brackets += source[0]
source = source[1:]
source = source[1:] # "]"
command, *args = inside_brackets.split(" ")
positional_args = []
named_args = {}
if "=" in command:
key, value = command.split("=", 1)
command = key
named_args[key] = try_convert_int(value)
for arg in args:
if "=" in arg:
key, value = arg.split("=", 1)
named_args[key.lower()] = try_convert_int(value.lower())
else:
positional_args.append(try_convert_int(arg))
return command.lower(), positional_args, named_args, source
def color_to_code(color, ctx="normal"):
COLORS = {
"normal": {
"normal": 0x0A,
"red": 0x20,
"pink": 0x21,
"purple": 0x22,
"blue": 0x23,
"cyan": 0x24,
"green": 0x25,
"yellow": 0x26,
},
"diary": {
"normal": 0x00,
"red": 0x07,
},
"inspect": {
"dark": 0x17,
},
"button": {
"blue": 0x10,
"green": 0x11,
"red": 0x12,
"yellow": 0x13,
"gray": 0x14,
"grey": 0x14,
},
"popup": {
"red": 0x28,
"pink": 0x29,
"purple": 0x2A,
"blue": 0x2B,
"teal": 0x2C,
"green": 0x2D,
"yellow": 0x2E,
"normal": 0x2F,
},
"sign": {
"normal": 0x18,
"red": 0x19,
"blue": 0x1A,
"green": 0x1B,
}
}
if type(color) is int:
return color
return COLORS.get(ctx, {}).get(color)
CHARSET = {
"𝅘𝅥𝅮": 0x00,
"!": 0x01,
'"': 0x02,
"#": 0x03,
"$": 0x04,
"%": 0x05,
"&": 0x06,
"'": 0x07,
"(": 0x08,
")": 0x09,
"*": 0x0A,
"+": 0x0B,
",": 0x0C,
"-": 0x0D,
".": 0x0E,
"/": 0x0F,
"0": 0x10,
"1": 0x11,
"2": 0x12,
"3": 0x13,
"4": 0x14,
"5": 0x15,
"6": 0x16,
"7": 0x17,
"8": 0x18,
"9": 0x19,
":": 0x1A,
";": 0x1B,
"<": 0x1C,
"=": 0x1D,
">": 0x1E,
"?": 0x1F,
"@": 0x20,
"A": 0x21,
"B": 0x22,
"C": 0x23,
"D": 0x24,
"E": 0x25,
"F": 0x26,
"G": 0x27,
"H": 0x28,
"I": 0x29,
"J": 0x2A,
"K": 0x2B,
"L": 0x2C,
"M": 0x2D,
"N": 0x2E,
"O": 0x2F,
"P": 0x30,
"Q": 0x31,
"R": 0x32,
"S": 0x33,
"T": 0x34,
"U": 0x35,
"V": 0x36,
"W": 0x37,
"X": 0x38,
"Y": 0x39,
"Z": 0x3A,
"[": 0x3B,
"¥": 0x3C,
"]": 0x3D,
"^": 0x3E,
"_": 0x3F,
"`": 0x40,
"a": 0x41,
"b": 0x42,
"c": 0x43,
"d": 0x44,
"e": 0x45,
"f": 0x46,
"g": 0x47,
"h": 0x48,
"i": 0x49,
"j": 0x4A,
"k": 0x4B,
"l": 0x4C,
"m": 0x4D,
"n": 0x4E,
"o": 0x4F,
"p": 0x50,
"q": 0x51,
"r": 0x52,
"s": 0x53,
"t": 0x54,
"u": 0x55,
"v": 0x56,
"w": 0x57,
"x": 0x58,
"y": 0x59,
"z": 0x5A,
"{": 0x5B,
"|": 0x5C,
"}": 0x5D,
"~": 0x5E,
"°": 0x5F,
"À": 0x60,
"Á": 0x61,
"Â": 0x62,
"Ä": 0x63,
"Ç": 0x64,
"È": 0x65,
"É": 0x66,
"Ê": 0x67,
"Ë": 0x68,
"Ì": 0x69,
"Í": 0x6A,
"Î": 0x6B,
"Ï": 0x6C,
"Ñ": 0x6D,
"Ò": 0x6E,
"Ó": 0x6F,
"Ô": 0x70,
"Ö": 0x71,
"Ù": 0x72,
"Ú": 0x73,
"Û": 0x74,
"Ü": 0x75,
"ß": 0x76,
"à": 0x77,
"á": 0x78,
"â": 0x79,
"ä": 0x7A,
"ç": 0x7B,
"è": 0x7C,
"é": 0x7D,
"ê": 0x7E,
"ë": 0x7F,
"ì": 0x80,
"í": 0x81,
"î": 0x82,
"ï": 0x83,
"ñ": 0x84,
"ò": 0x85,
"ó": 0x86,
"ô": 0x87,
"ö": 0x88,
"ù": 0x89,
"ú": 0x8A,
"û": 0x8B,
"ü": 0x8C,
"¡": 0x8D,
"¿": 0x8E,
"ª": 0x8F,
"": 0x90,
"": 0x91,
"": 0x92,
"": 0x93,
"": 0x94,
"": 0x95,
"": 0x96,
"": 0x97,
"": 0xA2,
"": 0xA3,
"": 0xA4,
"": 0xA5,
" ": 0xF7,
"": [0xFF, 0x24, 0xFF, 0x05, 0x10, 0x98, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x11, 0x99, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x12, 0xA1, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x13, 0x9D, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x13, 0x9E, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x13, 0x9F, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x13, 0xA0, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x14, 0x9A, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x14, 0x9B, 0xFF, 0x25],
"": [0xFF, 0x24, 0xFF, 0x05, 0x14, 0x9C, 0xFF, 0x25],
}
CHARSET_CREDITS = {
"A": 0x00,
"B": 0x01,
"C": 0x02,
"D": 0x03,
"E": 0x04,
"F": 0x05,
"G": 0x06,
"H": 0x07,
"I": 0x08,
"J": 0x09,
"K": 0x0A,
"L": 0x0B,
"M": 0x0C,
"N": 0x0D,
"O": 0x0E,
"P": 0x0F,
"Q": 0x10,
"R": 0x11,
"S": 0x12,
"T": 0x13,
"U": 0x14,
"V": 0x15,
"W": 0x16,
"X": 0x17,
"Y": 0x18,
"Z": 0x19,
"'": 0x1A,
".": 0x1B,
",": 0x1C,
"0": 0x1D,
"1": 0x1E,
"2": 0x1F,
"3": 0x20,
"4": 0x21,
"5": 0x22,
"6": 0x23,
"7": 0x24,
"8": 0x25,
"9": 0x26,
"©": 0x27,
"&": 0x28,
" ": 0xF7,
}
if __name__ == "__main__":
if len(argv) < 3:
print("usage: parse_compile.py [in.msg] [out.msgpack]")
exit(1)
_, filename, outfile = argv
messages = []
message = None
with open(filename, "r") as f:
source = f.read()
lineno = 1
charset = CHARSET
font_stack = [0]
sound_stack = [0]
color_stack = [0x0A]
while len(source) > 0:
if source.startswith("\n"):
lineno += 1
source = source[1:]
continue
if message is None:
if source.startswith("//"):
while source[0] != "\n":
source = source[1:]
else:
command, positional_args, named_args, source = parse_command(source)
if not command:
print(f"{filename}:{lineno}: expected [message]")
exit(1)
name = positional_args[0] if len(positional_args) > 0 else None
message = Message(name, named_args.get("section"), named_args.get("index"))
messages.append(message)
else:
command, positional_args, named_args, source = parse_command(source)
if command:
if command == "/message":
message.bytes += [0xFD]
# padding
while len(message.bytes) % 4 != 0:
message.bytes += [0x00]
message = None
elif command == "raw":
message.bytes += [*positional_args]
elif command == "func":
message.bytes += [0xFF, *positional_args]
elif command == "br":
message.bytes += [0xF0]
elif command == "prompt":
message.bytes += [0xF1]
elif command == "sleep":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: {command} command requires a positional parameter")
exit(1)
message.bytes += [0xF2, positional_args[0]]
elif command == "next":
message.bytes += [0xFB]
elif command == "color":
if "color" not in named_args:
print(f"{filename}:{lineno}: color command requires a 'color' parameter")
exit(1)
color = color_to_code(**named_args)
if color is None:
print(f"{filename}:{lineno}: unknown color combination {named_args}")
exit(1)
message.bytes += [0xFF, 0x05, color]
color_stack.append(color)
elif command == "/color":
color_stack.pop()
message.bytes += [0xFF, 0x05, color_stack[0]]
elif command == "style":
if "style" not in named_args:
print(f"{filename}:{lineno}: style command requires a 'style' parameter")
exit(1)
message.bytes += [0xFC]
style = named_args["style"]
if type(style) is int:
message.bytes += [style, *positional_args]
else:
if style == "right":
message.bytes += [0x01]
elif style == "left":
message.bytes += [0x02]
elif style == "center":
message.bytes += [0x03]
elif style == "tattle":
message.bytes += [0x04]
elif style == "choice":
if "w" not in named_args or "h" not in named_args or "x" not in named_args or "y" not in named_args:
print(f"{filename}:{lineno}: 'choice' style requires parameters: x, y, w, h")
exit(1)
message.bytes += [0x05, named_args["x"], named_args["y"], named_args["w"], named_args["h"]]
elif style == "inspect":
message.bytes += [0x06]
elif style == "sign":
message.bytes += [0x07]
elif style == "lamppost":
message.bytes += [0x08]
elif style == "postcard":
message.bytes += [0x09]
elif style == "popup":
message.bytes += [0x0A]
elif style == "upgrade":
if "w" not in named_args or "h" not in named_args or "x" not in named_args or "y" not in named_args:
print(f"{filename}:{lineno}: 'upgrade' style requires parameters: x, y, w, h")
exit(1)
message.bytes += [0x0C, named_args["w"], named_args["x"], named_args["h"], named_args["y"]]
elif style == "narrate":
message.bytes += [0x0D]
elif style == "epilogue":
message.bytes += [0x0E]
elif command == "font":
if "font" not in named_args:
print(f"{filename}:{lineno}: font command requires a 'font' parameter")
exit(1)
font = named_args["font"]
if font == "normal":
font = 0
elif font == "title":
font = 3
elif font == "subtitle":
font = 4
if type(font) is not int:
print(f"{filename}:{lineno}: unknown font '{font}'")
exit(1)
message.bytes += [0xFF, 0x00, font]
font_stack.append(font)
if font == 3 or font == 4:
charset = CHARSET_CREDITS
else:
charset = CHARSET
elif command == "/font":
font_stack.pop()
message.bytes += [0xFF, 0x00, font_stack[0]]
if font == 3 or font == 4:
charset = CHARSET_CREDITS
else:
charset = CHARSET
elif command == "noskip":
message.bytes += [0xFF, 0x07]
elif command == "/noskip":
message.bytes += [0xFF, 0x08]
elif command == "instant":
message.bytes += [0xFF, 0x09]
elif command == "/instant":
message.bytes += [0xFF, 0x0A]
elif command == "kerning":
if "kerning" not in named_args:
print(f"{filename}:{lineno}: kerning command requires a 'kerning' parameter")
exit(1)
message.bytes += [0xFF, 0x0B, named_args["kerning"]]
elif command == "scroll":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: scroll command requires a positional parameter")
exit(1)
message.bytes += [0xFF, 0x0C, positional_args[0]]
elif command == "size":
if "x" not in named_args or "y" not in named_args:
print(f"{filename}:{lineno}: size command requires parameters: x, y")
exit(1)
message.bytes += [0xFF, 0x0D, named_args["x"], named_args["y"]]
elif command == "/size":
message.bytes += [0xFF, 0x0E]
elif command == "speed":
if "delay" not in named_args or "chars" not in named_args:
print(f"{filename}:{lineno}: speed command requires parameters: delay, chars")
exit(1)
message.bytes += [0xFF, 0x0F, named_args["delay"], named_args["chars"]]
elif command == "pos":
if "y" not in named_args:
print(f"{filename}:{lineno}: pos command requires parameter: y (x is optional)")
exit(1)
if "x" in named_args:
message.bytes += [0xFF, 0x10, named_args["x"], named_args["y"]]
else:
message.bytes += [0xFF, 0x11, named_args["y"]]
elif command == "indent":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: indent command requires a positional parameter")
exit(1)
message.bytes += [0xFF, 0x12, positional_args[0]]
elif command == "down":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: down command requires a positional parameter")
exit(1)
message.bytes += [0xFF, 0x13, positional_args[0]]
elif command == "up":
if len(positional_args) == 0:
print(f"{filename}:{lineno}: up command requires a positional parameter")
exit(1)
message.bytes += [0xFF, 0x14, positional_args[0]]
elif command == "image":
if len(positional_args) == 1:
message.bytes += [0xFF, 0x15, positional_args[0]]
elif len(positional_args) == 7:
message.bytes += [0xFF, 0x18, *positional_args]
else:
print(f"{filename}:{lineno}: image command requires 1 or 7 positional parameters")
exit(1)
elif command == "sprite":
if len(positional_args) != 3:
print(f"{filename}:{lineno}: sprite command requires 3 positional parameters")
exit(1)
message.bytes += [0xFF, 0x16, *positional_args]
elif command == "item":
if len(positional_args) != 2:
print(f"{filename}:{lineno}: item command requires 2 positional parameters")
exit(1)
message.bytes += [0xFF, 0x17, *positional_args]
elif command == "cursor":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: cursor command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x1E, *positional_args]
elif command == "option":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: option command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x21, *positional_args]
elif command == "choice":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: choice command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x1E, positional_args[0], 0xFF, 0x21, positional_args[0]]
elif command == "choicecount":
if "choicecount" not in named_args:
print(f"{filename}:{lineno}: choicecount command requires a 'choicecount' parameter")
exit(1)
message.bytes += [0xFF, 0x1F, named_args["choicecount"]]
elif command == "cancel":
if "cancel" not in named_args:
print(f"{filename}:{lineno}: cancel command requires a 'cancel' parameter")
exit(1)
message.bytes += [0xFF, 0x20, named_args["cancel"]]
elif command == "shaky":
message.bytes += [0xFF, 0x26, 0x00]
elif command == "/shaky":
message.bytes += [0xFF, 0x27, 0x00]
elif command == "wavy":
message.bytes += [0xFF, 0x26, 0x01]
elif command == "/wavy":
message.bytes += [0xFF, 0x27, 0x01]
elif command == "shaky":
if "opacity" in named_args:
print(f"{filename}:{lineno}: shaky command doesn't accept parameter 'fade' (hint: did you mean 'faded-shaky'?)")
exit(1)
message.bytes += [0xFF, 0x26, 0x00]
elif command == "/shaky":
message.bytes += [0xFF, 0x27, 0x00]
elif command == "noise":
message.bytes += [0xFF, 0x26, 0x03, named_args.get("fade", 3)]
elif command == "/noise":
message.bytes += [0xFF, 0x27, 0x03]
elif command == "faded-shaky":
message.bytes += [0xFF, 0x26, 0x05, named_args.get("fade", 5)]
elif command == "/faded-shaky":
message.bytes += [0xFF, 0x27, 0x05]
elif command == "fade":
message.bytes += [0xFF, 0x26, 0x07, named_args.get("fade", 7)]
elif command == "/fade":
message.bytes += [0xFF, 0x27, 0x07]
elif command == "shout" or command == "shrinking":
message.bytes += [0xFF, 0x26, 0x0A]
elif command == "/shout" or command == "/shrinking":
message.bytes += [0xFF, 0x27, 0x0A]
elif command == "whisper" or command == "growing":
message.bytes += [0xFF, 0x26, 0x0B]
elif command == "/whisper" or command == "/growing":
message.bytes += [0xFF, 0x27, 0x0B]
elif command == "scream" or command == "shaky-size":
message.bytes += [0xFF, 0x26, 0x0C]
elif command == "/scream" or command == "/shaky-size":
message.bytes += [0xFF, 0x27, 0x0C]
elif command == "chortle" or command == "wavy-size":
message.bytes += [0xFF, 0x26, 0x0D]
elif command == "/chortle" or command == "/wavy-size":
message.bytes += [0xFF, 0x27, 0x0D]
elif command == "shadow":
message.bytes += [0xFF, 0x26, 0x0E]
elif command == "/shadow":
message.bytes += [0xFF, 0x27, 0x0E]
elif command == "var":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: var command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x28, *positional_args]
elif command == "center":
if len(positional_args) != 1:
print(f"{filename}:{lineno}: center command requires 1 positional parameter")
exit(1)
message.bytes += [0xFF, 0x29, *positional_args]
elif command == "volume":
if "volume" not in named_args:
print(f"{filename}:{lineno}: volume command requires a 'volume' parameter")
exit(1)
message.bytes += [0xFF, 0x2E, named_args["volume"]]
elif command == "sound":
if "sound" not in named_args:
print(f"{filename}:{lineno}: sound command requires a 'sound' parameter")
exit(1)
sound = named_args["sound"]
if sound == "normal":
sound = 0
elif sound == "bowser":
sound = 1
elif sound == "spirit":
sound = 2
if type(sound) is not int:
print(f"{filename}:{lineno}: unknown sound '{sound}'")
exit(1)
message.bytes += [0xFF, 0x2F, sound]
sound_stack.append(sound)
elif command == "/sound":
sound_stack.pop()
message.bytes += [0xFF, 0x2F, sound_stack[0]]
elif command == "a":
color_code = color_to_code(named_args.get("color", "blue"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x98, 0xFF, 0x25]
elif command == "b":
color_code = color_to_code(named_args.get("color", "green"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x99, 0xFF, 0x25]
elif command == "l":
color_code = color_to_code(named_args.get("color", "gray"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9A, 0xFF, 0x25]
elif command == "r":
color_code = color_to_code(named_args.get("color", "gray"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9B, 0xFF, 0x25]
elif command == "z":
color_code = color_to_code(named_args.get("color", "gray"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9C, 0xFF, 0x25]
elif command == "c-up":
color_code = color_to_code(named_args.get("color", "yellow"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9D, 0xFF, 0x25]
elif command == "c-down":
color_code = color_to_code(named_args.get("color", "yellow"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9E, 0xFF, 0x25]
elif command == "c-left":
color_code = color_to_code(named_args.get("color", "yellow"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0x9F, 0xFF, 0x25]
elif command == "c-right":
color_code = color_to_code(named_args.get("color", "yellow"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0xA0, 0xFF, 0x25]
elif command == "start":
color_code = color_to_code(named_args.get("color", "red"), named_args.get("ctx", "button"))
message.bytes += [0xFF, 0x24, 0xFF, 0x05, color_code, 0xA1, 0xFF, 0x25]
elif command == "note":
message.bytes += [0x00]
elif command == "heart":
message.bytes += [0x90]
elif command == "star":
message.bytes += [0x91]
elif command == "arrow-up":
message.bytes += [0x92]
elif command == "arrow-down":
message.bytes += [0x93]
elif command == "arrow-left":
message.bytes += [0x94]
elif command == "arrow-right":
message.bytes += [0x95]
elif command == "circle":
message.bytes += [0x96]
elif command == "cross":
message.bytes += [0x97]
elif command == "wait":
print(f"{filename}:{lineno}: unknown command 'wait' (hint: did you mean 'prompt'?)")
exit(1)
elif command == "pause":
print(f"{filename}:{lineno}: unknown command 'pause' (hint: did you mean 'sleep'?)")
exit(1)
else:
print(f"{filename}:{lineno}: unknown command '{command}'")
exit(1)
else:
if source[0] == "\\":
source = source[1:]
if source[0] in charset:
data = charset[source[0]]
if type(data) is int:
message.bytes.append(data)
else:
message.bytes += data
source = source[1:]
else:
print(f"{filename}:{lineno}: unsupported character '{source[0]}' for current font")
exit(1)
if message != None:
print(f"{filename}: missing [/message]")
exit(1)
with open(outfile, "wb") as f:
msgpack.pack([{
"section": message.section,
"index": message.index,
"name": message.name,
"bytes": bytes(message.bytes),
} for message in messages], f)

Binary file not shown.

View File

@ -0,0 +1 @@
build_system = "ninja"

View File

@ -1,2 +0,0 @@
REM first set your default distro using: wsl --set-default <Distro>
wt --title "diff.py" -d "../" ; split-pane -d "../" -V ; new-tab --title "mips_to_c.py" -d "../" ; focus-tab -t 0