What my workself would love is to easily dump Pandas or Polar data frames to SQL Tables in SQL Server as fast as possible. I see this bcp, but I don't see an example of uploading a large panda dataframe to SQL Server.
[deleted]
Microsoft Python Driver for SQL Server
21–30 of 35 posts
Re: Microsoft Python Driver for SQL Server
#22What my workself would love is to easily dump Pandas or Polar data frames to SQL Tables in SQL Server as fast as possible. I see this bcp, but I don't see an example of uploading a large panda dataframe to SQL Server.
On BULK INSERT, the data source doesn't have to live on the actual MSSQL box. It can be something on a UNC or even in Azure:
https://learn.microsoft.com/en-us/sql/t-sql/statements/bulk-...
> Beginning with SQL Server 2017 (14.x), the data_file can be in Azure Blob Storage.
You don't need to have access to the MSSQL box to invoke high performance batch loads. UNC should give you what you need if your IT department is willing to work with it.
Re: Microsoft Python Driver for SQL Server
#23Very cool. Used to be a huge pain to connect to sqlserver from Python (especially non Windows platforms).
I have very limited Python experience but I remember using pymssql in the past. Are there any problems with it?
We used it heavily 10 years ago. It was okay, but it had a rocky history. For a long time it seemed abandoned, then Python 3 happened and we had to patch our own version for a while. Then a new maintainer took over and stuff would just break or APIs rewritten between versions. We ran our own pypi instance to deal with pymssql specifically.
In the later years it became rather good though and with Python 3 many of the issues with character encoding went away.
Re: Microsoft Python Driver for SQL Server
#24Oooh, has it got sql_variant support? And how far back does SQL Server compatibility go?
Re: Microsoft Python Driver for SQL Server
#25If MSSQL really wanted to become more mainstream they'd release a properly free version to compete with MySQL/MariaDB and PostgreSQL. I've not used MSSQL since 2015/2016 and haven't missed much. Now I live in the OLAP space so I think of it far, far less.
Money here means this won’t happen. Sure, greenfield does not use MSSQL but there is a ton of companies stuck with MSSQL that will continue to have to fork over big licensing money.
Which will mean it’ll stay irrelevant still.
Re: Microsoft Python Driver for SQL Server
#26Very cool. Used to be a huge pain to connect to sqlserver from Python (especially non Windows platforms).
I have very limited Python experience but I remember using pymssql in the past. Are there any problems with it?
iirc it had problems with named instances - only on some newer version of sql server
Re: Microsoft Python Driver for SQL Server
#27https://github.com/microsoft/mssql-python/blob/main/mssql_py...
Is this generated by LLM? Comments are straight out of generic LLM slop.
Re: Microsoft Python Driver for SQL Server
#28How is this different than pymssql?
Re: Microsoft Python Driver for SQL Server
#29Earlier quoted context omitted.
Money here means this won’t happen. Sure, greenfield does not use MSSQL but there is a ton of companies stuck with MSSQL that will continue to have to fork over big licensing money.
Which will mean it’ll stay irrelevant still.
Which Microsoft is fine with, they bought Postgres sharding company and Azure supports all the popular DBs so they still get their money.
Re: Microsoft Python Driver for SQL Server
#30Twenty years ago, when I first used Python 2.5 to connect to MS SQL Server, I had to rely on a library called adodbapi. Back then, I thought the future would move toward OLE DB—but it has since become obsolete.