One really convenient module I use all the time is the csv module. You would think that parsing csv by yourself is easy, i.e., if you've never tried it, but it's really rather tricky.
People who use excel (sneer) often need to send me csv-formatted spreadsheets, and quite often the formatting is quite messed up. Often the line-endings are bizarre and frightening, and unrecognizable.
The solution is to add a 'U' flag to the open() call to enable universal newline support.
For instance, if you're opening a file for reading:
open('somefile.csv', 'rU')
will not care what the newline format is. Somehow it manages it all without me having to think about it. My thoughts are expensive.
A warning about tiktoken, BPE, and OpenAI models
33 minutes ago
No comments:
Post a Comment