Canary 1b

[janjanusek]

Hello,
is it possible to ise scripts for nemo model conversion to convert this model?

https://huggingface.co/nvidia/canary-1b

I'm asking because it supports put of the mox also translation and other tasks but I'm okay to have only it asr capabilities.

Thank you

[csukuangfj]

Can you test if it works? We have not tried to do that since it is very large and if you run it on CPU, it would be very slow.

[janjanusek]

unfortunatelly no,

it crashed when I tried to convert to onnx

--------------------------------------------------------------------------- 
AttributeError                            Traceback (most recent call last)
Cell In[9], line 14
      1 # convert_nemo_file('stt_enes_conformer_ctc_large_codesw') # big multi
      2 
      3 # convert_nemo_file('stt_de_conformer_ctc_large') # big
   (...)
     12 # convert_nemo_file('stt_fr_conformer_ctc_large') # big
     13 # convert_nemo_file('stt_fr_quartznet15x5') # jr only
---> 14 convert_nemo_file('canary-1b') # jr 

Cell In[7], line 18
     16 model_export_path = f'{model}/model.onnx'
     17 print(f'exporting model {model} to onnx')
---> 18 m.export(model_export_path)
     19 print(f"Converted file saved as: {model}")
     20 tokens_path = f'{model}/tokens.txt'

File ~/miniconda3/envs/nemo/lib/python3.10/site-packages/nemo/core/classes/exportable.py:117, in Exportable.export(self, output, input_example, verbose, do_constant_folding, onnx_opset_version, check_trace, dynamic_axes, check_tolerance, export_modules_as_functions, keep_initializers_as_inputs, use_dynamo)
    115 model = self.get_export_subnet(subnet_name)
    116 out_name = augment_filename(output, subnet_name)
--> 117 out, descr, out_example = model._export(
    118     out_name,
    119     input_example=input_example,
    120     verbose=verbose,
...
-> 1931 raise AttributeError(
   1932     f"'{type(self).__name__}' object has no attribute '{name}'"
   1933 )

AttributeError: 'EncDecMultiTaskModel' object has no attribute 'output_names'

[dnhkng]

Any updates on this? This model looks extremely interesting, although the real time factor is not amazing.

@csukuangfj How about nvidia/parakeet-tdt-1.1b? This should run fast, on CPU as the real time factor is 10x higher than whisper turbo or whisper distil.

I believe it should be similar to convert as nvidia/parakeet-tdt_ctc-110m which you have already done.