![]() We only want the weights in our new or modified layers to be updating.Īfter we do this, all that's left is just to train the model on our new data. Were after being trained on the original task. We want to keep all of these weights the same as they ![]() Layers at the end of our model may have learned features that are very specific to the original task, where as layers at the start of the model usually learn more general features like edges, shapes, and textures.Īfter we've modified the structure of the existing model, we then want to freeze the layers in our new model that came from the original model.įreezing, we mean that we don't want the weights for these layers to update whenever we train the model on our new data for our new task. This will depend on how similar the task is for each of the models. In some problems, we may want to remove more than just the last single layer, and we may want to add more than just one layer. After removing this, we want to add a new layer back that's purpose is toĬlassify whether an image is a truck or not. The last layer would have previously been classifying whether an image was a car or not. Going back to the example we just mentioned, if we have a model that has already been trained to recognize cars and we want to fine-tune this model to recognize trucks, we can first import our original model that was used on the cars problem.įor simplicity purposes, let's say we remove the last layer of this model. So this sounds fantastic, right, but how do we actually technically implement this? All of these learned features are definitely things we could benefit from in our new model for ![]() This already trained model has learned to understand edges and shapes and textures and more objectively, head lights, door handles, windshields, tires, etc. However, think about everything our model for recognizing trucks could use from the model that was originally Now, of course, if the two tasks are different, then there will be some information that the model has learned that may not apply to our new task, or there may be new information that the model needs to learn from the data regarding the new task thatįor example, a model trained on cars is not going to have ever seen a truck bed, so this feature is something new the model would have to learn about. ![]() If we can find a trained model that already does one task well, and that task is similar to ours in at least some remote way, then we can take advantage of everything the model has already learned This is what makes the fine-tuning approach so attractive. ![]() When building a model from scratch, we usually must try many approaches through trial-and-error.įor example, we have to choose how many layers we're using, what types of layers we're using, what order to put the layers in, how many nodes to include in each layer, decide how much regularization to use, what to set our learning rateīuilding and validating our model can be a huge task in its own right, depending on what data we're training it on. Model to make it perform a second similar task.Īssuming the original task is similar to the new task, using an artificial neural network that has already been designed and trained allows us to take advantage of what the model has already learned without having to develop it from scratch. Specifically, fine-tuning is a process that takes a model that has already been trained for one given task and then tunes or tweaks the Transfer learning occurs when we use knowledge that was gained from solving one problem and apply it to a new but related problem.įor example, knowledge gained from learning to recognize cars could be applied in a problem of recognizing trucks.įine-tuning is a way of applying or utilizing transfer learning. ![]()
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |