Problems reproducing model export
I can't reproduce the export of the model as you did. I get the error: Some weights of the model checkpoint at lilt-distilroberta-base were not used when initializing LiltForTokenClassification. Trying with your model the error does not appear. The colab I'm testing with is the following: https://colab.research.google.com/drive/1k2uGoDBOQwrK4iokGJOQKdDfll0QPbl-#scrollTo=0ZmvE7ku4hSW.
Can you help me figure out what I'm doing wrong? Thanks in advance
Hi @Frank94 , i'm not quite sure what might be the error in your case, but I remember also having problems at first while creating a LiLT model using distilroberta.
If I recall correctly, here's two additional passage that I had to do:
- I had to manually download the pytorch_model.bin and config.json files from the hub because for some reasons using wget led me to some error
- I changed a couple of lines of the original get_weight_roberta_like.py script. In particular
https://github.com/jpWang/LiLT/blob/main/gen_weight_roberta_like.py#L19
config['model_type'] = 'LiltModel' #changed from 'liltrobertalike'
https://github.com/jpWang/LiLT/blob/main/gen_weight_roberta_like.py#L26
text_model = torch.load(opt.text, map_location=torch.device('cpu')) #changed to cpu
https://github.com/jpWang/LiLT/blob/main/gen_weight_roberta_like.py#L28
lilt_model = torch.load(opt.lilt, map_location=torch.device('cpu')) #changed to cpu
Last two changes were made because I was working on cpu, I don't know if that is also your case.
Hope this helps :)
Hi @Sennodipoi ,
I'll try to do some tests as you suggested.
thanks for your support :)