thallada/modmapper-web
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Don't terminate & recreate workers after each task

Browse Source 
After some testing, I'm fairly confident the issue I was seeing before isn't
happening now. After re-processing ~250 plugins a few times with the same
workers I was seeing significant slow-downs with large plugins. Also, if I was
listening to music in the browser it would start crackling and popping.

My hypothesis is that wee_alloc was causing a memory leak in the old version
which was causing browsers to freak out unless I recreated the workers on every
run. Though I'm not sure if that theory holds since I wasn't seeing out of
control memory usage in the dev tools. Perhaps wee_alloc just wasn't good at
managing memory after a certain number of allocate/deallocate cycles and was
putting a huge strain on the browser.

I'm hoping that reusing workers like this will also speed up processing and
possibly resolve some crashing that some users were seeing.
This commit is contained in:
Tyler Hallada 2022-09-26 22:59:42 -04:00
parent f8956413c2
commit 513e95d697
1 changed files with 3 additions and 12 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

15
lib/WorkerPool.ts
View File

@ -31,12 +31,6 @@ export class WorkerPool {
return this; return this;
} }
public async addWorker() {
const worker = await this.createWorker();
this.availableWorkers.push(worker);
this.assignWorker();
}
public async createWorker(): Promise<Worker> { public async createWorker(): Promise<Worker> {
return new Promise((resolve) => { return new Promise((resolve) => {
const worker = new Worker(new URL("../workers/PluginsLoader.worker.ts", import.meta.url)); const worker = new Worker(new URL("../workers/PluginsLoader.worker.ts", import.meta.url));
@ -49,12 +43,9 @@ export class WorkerPool {
} else if (typeof data !== "string") { } else if (typeof data !== "string") {
store.dispatch(decrementPending(1)); store.dispatch(decrementPending(1));
store.dispatch(addParsedPluginInOrder(data)); store.dispatch(addParsedPluginInOrder(data));
// Since web assembly memory cannot be shrunk, replace worker with a fresh one to avoid slow repeated
// invocations on the same worker instance. Repeated invocations are so slow that the delay in creating a this.availableWorkers.push(worker);
// new worker is worth it. In practice, there are usually more workers than tasks, so the delay does not slow this.assignWorker()
// down processing.
worker.terminate();
this.addWorker();
} }
}; };
}); });