Use generators for fetching large db record sets

Category: Python - Miscellaneous

When using the python DB API, it's tempting to always use a cursor's fetchall() method so that you can easily iterate through a result set. For very large result sets though, this could be expensive in terms of memory (and time to wait for the entire result set to come back). You can use fetchmany() instead, but then have to manage looping through the intemediate result sets. Here's a generator that simplifies that for you. Date: 18 April, 2012


Database

Homepage: http://code.activestate.com/recipes/137270-use-generators-for-fetching-large-db-record-sets/?in=lang-python

Developer: Christopher Prinos

License: Python License

Operating System: Windows

Add a Comment

all are required fields

     
What do you think of this resource?

Select Your Rate:

Votes:0

 

Related Scripts Download

phpMyAdmin is a tool written in PHP intended to handle the administration of MySQL over the Web.

developer Developer: phpMyAdmin devel team
license License: GNU General Public License (GPL)
operating systems Operating System: OS Independent


HSQLDB is the leading SQL relational database engine written in Java.

developer Developer: developer
license License: BSD License (original)
operating systems Operating System: All


Monaghan Consultants provide excellence in IT & Database consultancy, Web Hosting and Broadband connectivity.

developer Developer: Alex Monaghan
license License: GNU General Public License (GPL)
operating systems Operating System: Not Available


The Open Media Lending Database (OpenDb) is an extremely flexible application to catalogue all sorts of things including DVD, VCD, CD, VHS, GAMES, BOOKS & Laser Discs.

developer Developer: OpenDb
license License: GNU General Public License (GPL)
operating systems Operating System: Platform Independent


W-Agora (Web-Agora) is a professional database-driven communications system which allows you and your visitors to store and display messages, files, and other infos on your web site.

developer Developer: Marc Druilhe
license License: Freeware
operating systems Operating System: Unix, Windows


PluggedOut CMS is a powerful thin client "Content Management System".

developer Developer: Jonathan Beckett
license License: GNU General Public License (GPL)
operating systems Operating System: OS Independent


neo_my2pg is a Python script for migration from mysql to postgresql.

developer Developer: SourceForge.net
license License: Artistic License, GNU General Public License (GPL)
operating systems Operating System: Windows, Linux, Mac OS, BSD, Solaris


A quick way to find valid email addresses in text files using a regular expression search.

developer Developer: carl scharenberg
license License: Python License
operating systems Operating System: Windows


This class extends ConfigParser class to make it able to read and write configuration settings from specified registry key.

developer Developer: Ivan Begtin
license License: Python License
operating systems Operating System: Windows