orderinglist.py 14 KB

123456789101112131415161718192021222324252627282930313233343536373839404142434445464748495051525354555657585960616263646566676869707172737475767778798081828384858687888990919293949596979899100101102103104105106107108109110111112113114115116117118119120121122123124125126127128129130131132133134135136137138139140141142143144145146147148149150151152153154155156157158159160161162163164165166167168169170171172173174175176177178179180181182183184185186187188189190191192193194195196197198199200201202203204205206207208209210211212213214215216217218219220221222223224225226227228229230231232233234235236237238239240241242243244245246247248249250251252253254255256257258259260261262263264265266267268269270271272273274275276277278279280281282283284285286287288289290291292293294295296297298299300301302303304305306307308309310311312313314315316317318319320321322323324325326327328329330331332333334335336337338339340341342343344345346347348349350351352353354355356357358359360361362363364365366367368369370371372373374375376377378379380381382383384385386387388
  1. # ext/orderinglist.py
  2. # Copyright (C) 2005-2022 the SQLAlchemy authors and contributors
  3. # <see AUTHORS file>
  4. #
  5. # This module is part of SQLAlchemy and is released under
  6. # the MIT License: https://www.opensource.org/licenses/mit-license.php
  7. """A custom list that manages index/position information for contained
  8. elements.
  9. :author: Jason Kirtland
  10. ``orderinglist`` is a helper for mutable ordered relationships. It will
  11. intercept list operations performed on a :func:`_orm.relationship`-managed
  12. collection and
  13. automatically synchronize changes in list position onto a target scalar
  14. attribute.
  15. Example: A ``slide`` table, where each row refers to zero or more entries
  16. in a related ``bullet`` table. The bullets within a slide are
  17. displayed in order based on the value of the ``position`` column in the
  18. ``bullet`` table. As entries are reordered in memory, the value of the
  19. ``position`` attribute should be updated to reflect the new sort order::
  20. Base = declarative_base()
  21. class Slide(Base):
  22. __tablename__ = 'slide'
  23. id = Column(Integer, primary_key=True)
  24. name = Column(String)
  25. bullets = relationship("Bullet", order_by="Bullet.position")
  26. class Bullet(Base):
  27. __tablename__ = 'bullet'
  28. id = Column(Integer, primary_key=True)
  29. slide_id = Column(Integer, ForeignKey('slide.id'))
  30. position = Column(Integer)
  31. text = Column(String)
  32. The standard relationship mapping will produce a list-like attribute on each
  33. ``Slide`` containing all related ``Bullet`` objects,
  34. but coping with changes in ordering is not handled automatically.
  35. When appending a ``Bullet`` into ``Slide.bullets``, the ``Bullet.position``
  36. attribute will remain unset until manually assigned. When the ``Bullet``
  37. is inserted into the middle of the list, the following ``Bullet`` objects
  38. will also need to be renumbered.
  39. The :class:`.OrderingList` object automates this task, managing the
  40. ``position`` attribute on all ``Bullet`` objects in the collection. It is
  41. constructed using the :func:`.ordering_list` factory::
  42. from sqlalchemy.ext.orderinglist import ordering_list
  43. Base = declarative_base()
  44. class Slide(Base):
  45. __tablename__ = 'slide'
  46. id = Column(Integer, primary_key=True)
  47. name = Column(String)
  48. bullets = relationship("Bullet", order_by="Bullet.position",
  49. collection_class=ordering_list('position'))
  50. class Bullet(Base):
  51. __tablename__ = 'bullet'
  52. id = Column(Integer, primary_key=True)
  53. slide_id = Column(Integer, ForeignKey('slide.id'))
  54. position = Column(Integer)
  55. text = Column(String)
  56. With the above mapping the ``Bullet.position`` attribute is managed::
  57. s = Slide()
  58. s.bullets.append(Bullet())
  59. s.bullets.append(Bullet())
  60. s.bullets[1].position
  61. >>> 1
  62. s.bullets.insert(1, Bullet())
  63. s.bullets[2].position
  64. >>> 2
  65. The :class:`.OrderingList` construct only works with **changes** to a
  66. collection, and not the initial load from the database, and requires that the
  67. list be sorted when loaded. Therefore, be sure to specify ``order_by`` on the
  68. :func:`_orm.relationship` against the target ordering attribute, so that the
  69. ordering is correct when first loaded.
  70. .. warning::
  71. :class:`.OrderingList` only provides limited functionality when a primary
  72. key column or unique column is the target of the sort. Operations
  73. that are unsupported or are problematic include:
  74. * two entries must trade values. This is not supported directly in the
  75. case of a primary key or unique constraint because it means at least
  76. one row would need to be temporarily removed first, or changed to
  77. a third, neutral value while the switch occurs.
  78. * an entry must be deleted in order to make room for a new entry.
  79. SQLAlchemy's unit of work performs all INSERTs before DELETEs within a
  80. single flush. In the case of a primary key, it will trade
  81. an INSERT/DELETE of the same primary key for an UPDATE statement in order
  82. to lessen the impact of this limitation, however this does not take place
  83. for a UNIQUE column.
  84. A future feature will allow the "DELETE before INSERT" behavior to be
  85. possible, alleviating this limitation, though this feature will require
  86. explicit configuration at the mapper level for sets of columns that
  87. are to be handled in this way.
  88. :func:`.ordering_list` takes the name of the related object's ordering
  89. attribute as an argument. By default, the zero-based integer index of the
  90. object's position in the :func:`.ordering_list` is synchronized with the
  91. ordering attribute: index 0 will get position 0, index 1 position 1, etc. To
  92. start numbering at 1 or some other integer, provide ``count_from=1``.
  93. """
  94. from ..orm.collections import collection
  95. from ..orm.collections import collection_adapter
  96. __all__ = ["ordering_list"]
  97. def ordering_list(attr, count_from=None, **kw):
  98. """Prepares an :class:`OrderingList` factory for use in mapper definitions.
  99. Returns an object suitable for use as an argument to a Mapper
  100. relationship's ``collection_class`` option. e.g.::
  101. from sqlalchemy.ext.orderinglist import ordering_list
  102. class Slide(Base):
  103. __tablename__ = 'slide'
  104. id = Column(Integer, primary_key=True)
  105. name = Column(String)
  106. bullets = relationship("Bullet", order_by="Bullet.position",
  107. collection_class=ordering_list('position'))
  108. :param attr:
  109. Name of the mapped attribute to use for storage and retrieval of
  110. ordering information
  111. :param count_from:
  112. Set up an integer-based ordering, starting at ``count_from``. For
  113. example, ``ordering_list('pos', count_from=1)`` would create a 1-based
  114. list in SQL, storing the value in the 'pos' column. Ignored if
  115. ``ordering_func`` is supplied.
  116. Additional arguments are passed to the :class:`.OrderingList` constructor.
  117. """
  118. kw = _unsugar_count_from(count_from=count_from, **kw)
  119. return lambda: OrderingList(attr, **kw)
  120. # Ordering utility functions
  121. def count_from_0(index, collection):
  122. """Numbering function: consecutive integers starting at 0."""
  123. return index
  124. def count_from_1(index, collection):
  125. """Numbering function: consecutive integers starting at 1."""
  126. return index + 1
  127. def count_from_n_factory(start):
  128. """Numbering function: consecutive integers starting at arbitrary start."""
  129. def f(index, collection):
  130. return index + start
  131. try:
  132. f.__name__ = "count_from_%i" % start
  133. except TypeError:
  134. pass
  135. return f
  136. def _unsugar_count_from(**kw):
  137. """Builds counting functions from keyword arguments.
  138. Keyword argument filter, prepares a simple ``ordering_func`` from a
  139. ``count_from`` argument, otherwise passes ``ordering_func`` on unchanged.
  140. """
  141. count_from = kw.pop("count_from", None)
  142. if kw.get("ordering_func", None) is None and count_from is not None:
  143. if count_from == 0:
  144. kw["ordering_func"] = count_from_0
  145. elif count_from == 1:
  146. kw["ordering_func"] = count_from_1
  147. else:
  148. kw["ordering_func"] = count_from_n_factory(count_from)
  149. return kw
  150. class OrderingList(list):
  151. """A custom list that manages position information for its children.
  152. The :class:`.OrderingList` object is normally set up using the
  153. :func:`.ordering_list` factory function, used in conjunction with
  154. the :func:`_orm.relationship` function.
  155. """
  156. def __init__(
  157. self, ordering_attr=None, ordering_func=None, reorder_on_append=False
  158. ):
  159. """A custom list that manages position information for its children.
  160. ``OrderingList`` is a ``collection_class`` list implementation that
  161. syncs position in a Python list with a position attribute on the
  162. mapped objects.
  163. This implementation relies on the list starting in the proper order,
  164. so be **sure** to put an ``order_by`` on your relationship.
  165. :param ordering_attr:
  166. Name of the attribute that stores the object's order in the
  167. relationship.
  168. :param ordering_func: Optional. A function that maps the position in
  169. the Python list to a value to store in the
  170. ``ordering_attr``. Values returned are usually (but need not be!)
  171. integers.
  172. An ``ordering_func`` is called with two positional parameters: the
  173. index of the element in the list, and the list itself.
  174. If omitted, Python list indexes are used for the attribute values.
  175. Two basic pre-built numbering functions are provided in this module:
  176. ``count_from_0`` and ``count_from_1``. For more exotic examples
  177. like stepped numbering, alphabetical and Fibonacci numbering, see
  178. the unit tests.
  179. :param reorder_on_append:
  180. Default False. When appending an object with an existing (non-None)
  181. ordering value, that value will be left untouched unless
  182. ``reorder_on_append`` is true. This is an optimization to avoid a
  183. variety of dangerous unexpected database writes.
  184. SQLAlchemy will add instances to the list via append() when your
  185. object loads. If for some reason the result set from the database
  186. skips a step in the ordering (say, row '1' is missing but you get
  187. '2', '3', and '4'), reorder_on_append=True would immediately
  188. renumber the items to '1', '2', '3'. If you have multiple sessions
  189. making changes, any of whom happen to load this collection even in
  190. passing, all of the sessions would try to "clean up" the numbering
  191. in their commits, possibly causing all but one to fail with a
  192. concurrent modification error.
  193. Recommend leaving this with the default of False, and just call
  194. ``reorder()`` if you're doing ``append()`` operations with
  195. previously ordered instances or when doing some housekeeping after
  196. manual sql operations.
  197. """
  198. self.ordering_attr = ordering_attr
  199. if ordering_func is None:
  200. ordering_func = count_from_0
  201. self.ordering_func = ordering_func
  202. self.reorder_on_append = reorder_on_append
  203. # More complex serialization schemes (multi column, e.g.) are possible by
  204. # subclassing and reimplementing these two methods.
  205. def _get_order_value(self, entity):
  206. return getattr(entity, self.ordering_attr)
  207. def _set_order_value(self, entity, value):
  208. setattr(entity, self.ordering_attr, value)
  209. def reorder(self):
  210. """Synchronize ordering for the entire collection.
  211. Sweeps through the list and ensures that each object has accurate
  212. ordering information set.
  213. """
  214. for index, entity in enumerate(self):
  215. self._order_entity(index, entity, True)
  216. # As of 0.5, _reorder is no longer semi-private
  217. _reorder = reorder
  218. def _order_entity(self, index, entity, reorder=True):
  219. have = self._get_order_value(entity)
  220. # Don't disturb existing ordering if reorder is False
  221. if have is not None and not reorder:
  222. return
  223. should_be = self.ordering_func(index, self)
  224. if have != should_be:
  225. self._set_order_value(entity, should_be)
  226. def append(self, entity):
  227. super(OrderingList, self).append(entity)
  228. self._order_entity(len(self) - 1, entity, self.reorder_on_append)
  229. def _raw_append(self, entity):
  230. """Append without any ordering behavior."""
  231. super(OrderingList, self).append(entity)
  232. _raw_append = collection.adds(1)(_raw_append)
  233. def insert(self, index, entity):
  234. super(OrderingList, self).insert(index, entity)
  235. self._reorder()
  236. def remove(self, entity):
  237. super(OrderingList, self).remove(entity)
  238. adapter = collection_adapter(self)
  239. if adapter and adapter._referenced_by_owner:
  240. self._reorder()
  241. def pop(self, index=-1):
  242. entity = super(OrderingList, self).pop(index)
  243. self._reorder()
  244. return entity
  245. def __setitem__(self, index, entity):
  246. if isinstance(index, slice):
  247. step = index.step or 1
  248. start = index.start or 0
  249. if start < 0:
  250. start += len(self)
  251. stop = index.stop or len(self)
  252. if stop < 0:
  253. stop += len(self)
  254. for i in range(start, stop, step):
  255. self.__setitem__(i, entity[i])
  256. else:
  257. self._order_entity(index, entity, True)
  258. super(OrderingList, self).__setitem__(index, entity)
  259. def __delitem__(self, index):
  260. super(OrderingList, self).__delitem__(index)
  261. self._reorder()
  262. def __setslice__(self, start, end, values):
  263. super(OrderingList, self).__setslice__(start, end, values)
  264. self._reorder()
  265. def __delslice__(self, start, end):
  266. super(OrderingList, self).__delslice__(start, end)
  267. self._reorder()
  268. def __reduce__(self):
  269. return _reconstitute, (self.__class__, self.__dict__, list(self))
  270. for func_name, func in list(locals().items()):
  271. if (
  272. callable(func)
  273. and func.__name__ == func_name
  274. and not func.__doc__
  275. and hasattr(list, func_name)
  276. ):
  277. func.__doc__ = getattr(list, func_name).__doc__
  278. del func_name, func
  279. def _reconstitute(cls, dict_, items):
  280. """Reconstitute an :class:`.OrderingList`.
  281. This is the adjoint to :meth:`.OrderingList.__reduce__`. It is used for
  282. unpickling :class:`.OrderingList` objects.
  283. """
  284. obj = cls.__new__(cls)
  285. obj.__dict__.update(dict_)
  286. list.extend(obj, items)
  287. return obj