Is there any way to do Custom dynamic mapping of different number of columns in Azure dataflow or any other options to achieve this?

Question

My source (CSV file in ADLS) has header record(3 columns) , detail records(5 columns) and trailer record(2 columns) . The header record has less number of columns than the detail records. When I try to convert this csv file to parquet, i m getting the column count error using copy activity in ADF. So I tried using dataflow to do the mapping but still its considering only three columns and ignoring the other two columns in the detail records. So please let me know how to achieve this using dataflow or any other azure services.

Sample Data

1|~filename|~30122020
2|~Mark|~cse|~378|~2020
2|~John|~|~430|~2019
99|~3

Answer 1

Data factory will consider the first row as the column schema. Just for now, We can't do that with Data factory actives or Data Flow.