Exploring Local GPT Models: A Comparison of GPT-4 and Mistral 7B on WasmEdge
Comparing GPT-4 and (local) Mistral 7B: An comparison of their performance using WasmEdge for local GPT model execution.
I've been on the hunt for running GPT's locally for a bit and played around in the maze of Hugging Face models for awhile now.
The example provided on their post has the user ask "What is the capital of France" so after getting it set up I instead asked
[USER]: What is the capital of Texas? [ASSISTANT]: The capital of Texas is Austin. [USER]:
This took about 3 seconds, the hardware used was.
I didn't see a much GPU ussage and it seemed WasmEdge was only using around 4 cores.
Testing a bigger question with code
One of my todos for awhile has been automating opening tmux with all the commands I use to monitor my hardware during these sort of tests.
I need a shell script that opens up tmux with one large vertical window running htop on the left and two horizontal windows on the right each running an echo command I will replace.
- GPT4 took around 30 seconds
- Mistral took about 4 minutes and 50 seconds.
To create a shell script that opens tmux with a specific layout, you'll want to use tmux commands to split the window and run the desired commands in each pane. Here's a basic script that should achieve what you're looking for:
#!/bin/bash # Start a new tmux session tmux new-session -d -s my_session # Split the window vertically, 30% width for the left pane tmux split-window -h -p 70 # Split the right pane horizontally tmux split-window -v # Select the left pane and run htop tmux select-pane -t 0 tmux send-keys 'htop' C-m # Select the first right pane (top right) tmux select-pane -t 1 tmux send-keys 'echo Replace this command with your desired command' C-m # Select the second right pane (bottom right) tmux select-pane -t 2 tmux send-keys 'echo Replace this with another command' C-m # Attach to the tmux session tmux attach-session -t my_session
Here's a breakdown of what this script does:
- Creates a new tmux session named
- Splits the window vertically with 30% of the width allocated to the left pane (where
- Splits the right pane horizontally into two equal parts.
- Selects the left pane and runs
- Selects the top right pane and runs an
command (you can replace this with your desired command).
- Selects the bottom right pane and runs another
command (also replaceable).
- Attaches to the tmux session so you can interact with it.
To use this script:
- Save it to a file, for example,
- Make the script executable:
chmod +x tmux_setup.sh
- Run the script:
Remember to replace the
commands with your own commands as needed.
Mistral 7b instruct v0.1
[USER]: I need a shell script that opens up tmux with one large vertical window running htop on the left and two horizontal windows on the right each running an echo command I will replace.
[ASSISTANT]: Here is a shell script that will open up tmux with one large vertical window running htop on the left and two horizontal windows on the right each running an echo command:
#!/bin/bash # Open tmux tmux new -s my-session # Set the window layout tmux set-window-layout 100% 50% 50% # Start htop in the left window tmux send -t 1 "htop" # Start echo commands in the right windows tmux send -t 2 "echo Hello, world!" tmux send -t 3 "echo This is a test."
To use this script, save it to a file (e.g.
) and make it executable with the command
. Then, you can run the script with the command
chmod +x start_tmux.sh
. This will open tmux with the specified window layout and start the specified commands in the windows.