diff --git a/CHANGELOG.md b/CHANGELOG.md index 01ad672..9697787 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,5 +1,12 @@ # CHANGELOG +## v0.2.6 - 2026-05-01 + + * Fix grammar and typos in the documentation + * Fix example code in `PgLargeObjects.export/3` API docs + * Fix admonition title in `PgLargeObjects.LargeObject.size/1` API docs + * Fix potential resource leak in `PgLargeObjects.import/3` and `export/3`. + ## v0.2.5 - 2026-04-27 * Fix `PgLargeObjects.export/3` not honoring `:bufsize` option if `:into` diff --git a/CONSIDERATIONS.md b/CONSIDERATIONS.md index db8ae02..47f6ed0 100644 --- a/CONSIDERATIONS.md +++ b/CONSIDERATIONS.md @@ -8,9 +8,9 @@ PostgreSQL 4.2 was released on Jun 30th, 1994. The large objects facility has been around for a long time! Yet, it is fairly unknown to many programmers - or considered too unwieldy to -use for productive usage. This is not by accident - there are various trade -offs to consider when deciding if large objects are a good mechanism for -storing large amounts of binary data. +use for productive usage. This is not by accident - there are various trade-offs +to consider when deciding if large objects are a good mechanism for storing +large amounts of binary data. This document attempts to collect and discuss some of these considerations. If you feel there are other aspects to highlight, or if any of the items below @@ -41,14 +41,14 @@ mechanisms for storing large objects. For example, the [AWS RDS documentation](https://aws.amazon.com/rds/postgresql/pricing/) (RDS is Amazon's managed database offering) explains that at the time of this writing, 1GB of -General Purpose storage for a in the us-east-1 region costs $0.115 per month +General Purpose storage in the `us-east-1` region costs $0.115 per month for a PostgreSQL database. The [AWS S3 documentation](https://aws.amazon.com/s3/pricing/) (S3 is Amazon's object storage offering) documents, at the time of this writing, that storing -1GB of data in the us-east-1 region is a mere $0.023 per month! +1GB of data in the `us-east-1` region is a mere $0.023 per month! -I.e. when using Amazon cloud services in the us-east-1 region, storing data in -RDS is five times as expensive as storing it in S3. Depending on the amount of -data and your budget, this might be a significant difference. +I.e. when using Amazon cloud services in the `us-east-1` region, storing data +in RDS is five times as expensive as storing it in S3. Depending on the amount +of data and your budget, this might be a significant difference. Make sure to check the pricing (if applicable) for storage used by your PostgreSQL database and consider the change in the decision whether to use @@ -73,6 +73,6 @@ frerich@Mac ~ % pg_dump --help ``` Consider your current backup mechanism and see if it's configured to include or -exclude large objects. Decide on the important of large objects for your use +exclude large objects. Decide on the importance of large objects for your use case and include that in your decision on how often large objects should be included in backups. diff --git a/README.md b/README.md index bf6f75f..655f1f5 100644 --- a/README.md +++ b/README.md @@ -28,7 +28,7 @@ options for doing so: 2. A separate cloud storage (e.g. AWS S3) could be used. This permits streaming but requires complicating the tech stack by depending on a new service. Bridging the two systems (e.g. ‘Delete all uploads for a given user ID’) - requires Elixir support. + requires custom application code. PostgreSQL features a ‘large objects’ facility which enables efficient streaming access to large (up to 4TB) files. This solves these problems: @@ -39,9 +39,9 @@ streaming access to large (up to 4TB) files. This solves these problems: with the tables referencing them, operations like ‘Delete all uploads for a given user ID’ are just one `SELECT` statement. -However, there are trade offs. See the [Considerations](CONSIDERATIONS.md) +However, there are trade-offs. See the [Considerations](CONSIDERATIONS.md) document for aspects to take into account when deciding if large objects -are good choice for your use case. +are a good choice for your use case. ## Installation @@ -108,7 +108,7 @@ end Use the high-level APIs `PgLargeObjects.import/3` and `PgLargeObjects.export/3` (exposed as `import_large_object/2` and `export_large_object/2` on the - applications' repository module) for importing data into or exporting data out + application's repository module) for importing data into or exporting data out of the database: ```elixir diff --git a/lib/pg_large_objects.ex b/lib/pg_large_objects.ex index 13a047a..81df87b 100644 --- a/lib/pg_large_objects.ex +++ b/lib/pg_large_objects.ex @@ -38,9 +38,12 @@ defmodule PgLargeObjects do case data do binary when is_binary(binary) -> {:ok, buffer} = StringIO.open(binary, encoding: :latin1) - result = import(repo, IO.binstream(buffer, opts[:bufsize]), opts) - StringIO.close(buffer) - result + + try do + import(repo, IO.binstream(buffer, opts[:bufsize]), opts) + after + StringIO.close(buffer) + end enumerable -> with {:ok, lob} <- LargeObject.create(repo) do @@ -57,15 +60,15 @@ defmodule PgLargeObjects do Export data out of large object. This exports the data in the large object referenced by the object ID `oid`. - Depending on the `:into` option, the data is returned a single binary or fed - into a given `Collectable`. + Depending on the `:into` option, the data is returned as a single binary or + fed into a given `Collectable`. To treat a large object as an `Enumerable` and pass it around as a stream, reach for the lower-level API in `PgLargeObjects.LargeObject`, e.g.: ```elixir - def stream_object!(object_id) do - {:ok, object} = PgLargeObject.LargeObject.open(object_id) + def stream_object!(repo, object_id) do + {:ok, object} = PgLargeObjects.LargeObject.open(repo, object_id) object end ``` @@ -96,17 +99,19 @@ defmodule PgLargeObjects do {:ok, buffer} = StringIO.open("", encoding: :latin1) result = - with :ok <- - export(repo, oid, - into: IO.binstream(buffer, opts[:bufsize]), - bufsize: opts[:bufsize] - ) do - {_input, output} = StringIO.contents(buffer) - {:ok, output} + try do + with :ok <- + export(repo, oid, + into: IO.binstream(buffer, opts[:bufsize]), + bufsize: opts[:bufsize] + ) do + {_input, output} = StringIO.contents(buffer) + {:ok, output} + end + after + StringIO.close(buffer) end - StringIO.close(buffer) - result collectable -> diff --git a/lib/pg_large_objects/large_object.ex b/lib/pg_large_objects/large_object.ex index 242c330..2affbd8 100644 --- a/lib/pg_large_objects/large_object.ex +++ b/lib/pg_large_objects/large_object.ex @@ -13,7 +13,7 @@ defmodule PgLargeObjects.LargeObject do > #### Transactions Required {: .info} > > All operations on `LargeObject` values *must* take place within a database - > transactions since the internal handle managed by the structure is only + > transaction since the internal handle managed by the structure is only > valid for the duration of a transaction. > > Any large object value will be closed automatically at the end of the @@ -186,7 +186,7 @@ defmodule PgLargeObjects.LargeObject do Calculates the size (in bytes) of the given large object `lob`. - > #### Enum.count/1 vs. Enum.size/1 {: .info} + > #### Enum.count/1 vs. LargeObject.size/1 {: .info} > > Note that this is not the same as using `Enum.count/1`; `Enum.count/1`, by > virtue of the `Enumerable` implementation, will return the number of _chunks_ @@ -246,8 +246,8 @@ defmodule PgLargeObjects.LargeObject do @doc """ Read data from large object. - Reads a `length` bytes of data from the given large object `lob`, starting at - the current iosition in the object. Advanced the position by the number of + Reads `length` bytes of data from the given large object `lob`, starting at + the current position in the object. Advances the position by the number of bytes read, or until the end of file. The read position will not be advanced when the current position is beyond the end of the file. diff --git a/test/pg_large_objects/large_object_test.exs b/test/pg_large_objects/large_object_test.exs index 709abb6..b190af3 100644 --- a/test/pg_large_objects/large_object_test.exs +++ b/test/pg_large_objects/large_object_test.exs @@ -32,7 +32,7 @@ defmodule PgLargeObjects.LargeObjectTest do end test "fails given invalid object ID" do - assert {:error, :not_found} == LargeObject.open(TestRepo, 12_345) + assert {:error, :not_found} == LargeObject.remove(TestRepo, 12_345) end end @@ -208,7 +208,8 @@ defmodule PgLargeObjects.LargeObjectTest do assert {:ok, 1} = LargeObject.seek(lob, 0, :current) assert {:ok, "B"} == LargeObject.read(lob, 1) - # Seeting to 0 bytes from the end moves the cursor one past the last byte. + # Setting to 0 bytes from the end moves the cursor one past the last + # byte. assert {:ok, 7} = LargeObject.seek(lob, 0, :end) assert {:ok, ""} == LargeObject.read(lob, 1) end)