How to parse and load data into a database?

Question

Python

Given:
1. csv file, but relative to the curve. The data is present there in the format
field1, field2, field3, field4
data1, data2, data3, data4
field1, field2, field3, field4, field5
data1, data2, data3, data4, data5 some_line_with_random_data
!^*@$!( The
file describes the data of several tables at once, along the way there are blotches of just text strings of the subtotal type 2.
It is necessary to parse the data and store it in the database. 1. I
sketched
a function that runs through the entire file and extracts the lines I need, returns a list of the format
['data1,data2,data3\n', 'data4,data5,data6\n', ]
This is how the function looks like

spoiler

def parsecsv_equities(filename):
    f = open(filename, "r")

    trade_data_list = []


    for line in f:
        if 'Trades,Data,Order,Equity' in line:
            trade_data_list.append(line)



    f.close()

    return trade_data_list


def parsecsv_futures(filename):
    f = open(filename, "r")

    trade_data_list = []


    for line in f:
        if 'Trades,Data,Order,Futures' in line:
            trade_data_list.append(line)



    f.close()

    return trade_data_list

2. Wrote the skeleton of a function that connects to the database and quite successfully pulls requests. Until I fooled around with the problems of sql injections, this is secondary to me.

spoiler

import pyodbc


def dbconnect(str):
    conn = pyodbc.connect('Driver={SQL Server};'
                          'Server=1.1.1.1;'
                          'Database=Reports;'
                          'Trusted_Connection=yes;')

    cursor = conn.cursor()
    cursor.execute(str)

    for row in cursor:
        print(row)

And here I stood up. I just have no idea how to extract the data I need from the sheet and push it into the request. The peculiarity of the data is that the lines can be of different lengths, because the last field is optional.
Poke a noob’s nose on how it’s done, online courses are already starting to tell me about OOP, which is a fierce overkill for my task, but they didn’t really explain how to work with strings ... (well, or I’m stupid)

Answer 1

Can it is better to look at once towards Data Transformation Service (type an integral part of MS SQL)?

Answer 2

And you look at PIVOT
Either on the official Microsoft
documentation And in general, I would get a normal TABLE, and I would already break it into the necessary parts and transfer it to a new database.
A would have received a transposition, as in the above functions.