2022-2-12: Generating training data, EfficientNet-X, Editing factual knowledge
dblalock.substack.com
Locating and Editing Factual Knowledge in GPT They edit factual knowledge in GPT-J, meaning they, e.g., get the model to generate sentences as if the Eiffel tower were in Rome rather than Paris. Find-and-replace for inputs like the Madry paper, but they do this at the token level instead. Rank-1 update of one layer’s weight matrix is nice, but they have to optimize the "replacement" embedding via backprop, so I doubt there's much speedup to be had.
2022-2-12: Generating training data, EfficientNet-X, Editing factual knowledge
2022-2-12: Generating training data…
2022-2-12: Generating training data, EfficientNet-X, Editing factual knowledge
Locating and Editing Factual Knowledge in GPT They edit factual knowledge in GPT-J, meaning they, e.g., get the model to generate sentences as if the Eiffel tower were in Rome rather than Paris. Find-and-replace for inputs like the Madry paper, but they do this at the token level instead. Rank-1 update of one layer’s weight matrix is nice, but they have to optimize the "replacement" embedding via backprop, so I doubt there's much speedup to be had.