Something was adding whole seconds to transactions’ latencies.ĭigging through the SQLite source code, I looked for places where it could sleep in whole-second increments. I saw transactions that took 1.02 and 1.04 seconds, but never 1.61 seconds or 0.98 seconds. Every long-lasting transaction took slightly more than an integral number of seconds. And, eventually, some errant transaction would take 5.04 seconds and beets would crash: database is locked.īut there was a pattern. But, this time, an occasional transaction would sometimes take much longer: 1.08 seconds, say. And again, most transactions were in the one- or two-millisecond range. I again set about measuring the length of each transaction. Eventually, one incredibly helpful user offered to give me guest SSH access so I could see the bug manifest in vitro on his machine. I finally gave up trying to reproduce the problem on my own machine. I measured the length of each access and, on my machine, they each took a handful of milliseconds apiece-nowhere near a full five seconds. But all this was to no avail-the bug reports continued to pour in.Īt this point, I was almost certain that nothing was wrong with beets’ transactions in themselves. I fastidiously closed every cursor after each SELECT. I made extra-double-sure that filesystem operations happened only outside of transactions. So I looked at every line in the source where a transaction could start. No amount of SELECTs and INSERTs at beets’ scale should add up to five seconds, so turning up the timeout parameter is really just painting over the rot. That beets spends 5,000 milliseconds manipulating the database in a single transaction is indicative of something dark and terrible. But herein lies the mystery: five seconds is a long time. So the solution should be simple: somewhere, beets is holding a transaction open for more than five seconds, so we can either find the offending transaction or crank up that timeout. If it ever sees that a thread has been waiting for a lock for more than five seconds (by default), it throws up its hands and the user sees the dreaded database is locked error. For exactly this reason, SQLite has a lock timeout built in. If a transaction stays open too long, it can block other threads from accessing the database-or, in the worst case, several threads can deadlock while waiting for each other. Fortunately, SQLite’s transactions and ACID guarantees make this straightforward: each thread gets to make atomic accesses without bothering the other threads.īut things can go wrong. When importing music, multiple threads collaborate to speed up the process and several of the threads have to read from and write to the database. The ProblemĪ little bit of background: beets uses the amazing SQLite database library to store its music catalog. Like appendicitis, OperationalError can strike at any time, which makes it all the more maddening. This is particularly frustrating because there’s no correlation at all between what you do as a user and when this exception comes up. Sqlite3.OperationalError: database is locked
0 Comments
Leave a Reply. |
AuthorWrite something about yourself. No need to be fancy, just an overview. ArchivesCategories |