BC Libraries Coop wiki

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== BC Services Card Research Project ======

{{ :public:public_-_covid_recovery_fund_project_-_bc_services_card_patron_authentication_integration.pdf |Original Proposal}}

===== Patron information in Evergreen =====

Patrons are normally registered via the EG staff client, by staff at the circ desk ([[https://docs.evergreen-ils.org/eg/docs/latest/circulation/circulation_patron_records_web_client.html#_registering_new_patrons|docs]]). We also have custom scripts for importing patron records from student management systems for a few post-secondary libraries. There's no official API or process specifically intended to enable third-party applications to create/import/validate patron records (EG needs more third-party-friendly APIs generally), but there's a feature request to include a standard import script: [[https://bugs.launchpad.net/evergreen/+bug/1786524|LP#1786524]]

Several other Evergreen consortia (PINES and KCLS) use Quipu's eCARD product which has some overlap with BC Services Card use cases. There's been some effort to bring their Quipu integration into mainline Evergreen, ideally in a generic way that could also support similar products: [[https://bugs.launchpad.net/evergreen/+bug/1902937|LP#1902937]] (there's some discussion here of account renewal requirements at non-Sitka libraries which may be of interest too).

Evergreen also has a patron self-registration feature ([[https://docs.evergreen-ils.org/eg/docs/latest/circulation/circulation_patron_records_web_client.html#_patron_self_registration|main docs]] / [[https://docs.evergreen-ils.org/eg/docs/latest/admin/patron_self_registration.html|admin docs]]). <del>We're not using it, but</del> it's a good fit for this project; if nothing else, it provides a model for how to go about creating patron-initiated "pending" accounts in EG.

There's also a feature request for patron address validation using third-party services: [[https://bugs.launchpad.net/evergreen/+bug/1569889|LP#1569889]]

There is a lack of good documentation on how external applications can use EG's existing APIs, in part because those APIs were primarily designed to be used by EG's own component services. It would be worth investing in better documentation (and more user-friendly APIs, especially for creating/updating patron accounts).

==== Technical details ====

Here's an entity relationship diagram for the database tables that store patron information (click to enlarge):

{{ :public:actor-erd-v2.png?direct&200 |}}

A few notes:
  * actor.usr is the primary table for personal information. It is used for both staff and patron accounts.
  * Each user has a primary card (actor.usr.card) and may also have secondary cards, all of which are in actor.card.
  * Each user also has a username. Usually this matches the barcode of the patron's primary card, but not always.
  * The user may have a mailing address and a billing address, but they're not required. These are found in the actor.usr_address table. Multiple users may share the same address.
  * Email address is not required, nor guaranteed to be valid or well-formed. However, users can reset their password by clicking a link in the public catalogue which sends a reset email to whatever email address is in actor.usr.email.
  * The ident_value and ident_value2 fields may contain driver's license number, student/employee ID (for post-secondary patrons), or other information. These fields are uncontrolled, but we do use them as a matchpoint when importing patron data from student management systems, etc.
  * actor.usr.profile contains the user's profile, a.k.a. permission group or patron type. Patron and staff permissions are generally governed by which permission group they belong to.
  * actor.usr.home_ou specifies the user's home library. The hierarchy of libraries (branches, systems, federations, etc.) is defined in actor.org_unit.
  * Libraries can track additional information about their patrons using "statistical categories," which are found in actor.stat_cat and related tables. The library may use this to capture information not otherwise tracked in Evergreen, such as municipality or electoral district, residency status, student type, or gender. Each library defines its own categories and the permitted values for each category (the value is selected from a drop-down list, not a free text field).
  * When patron self-registration is permitted, the data entered by the patron is initially stored in the staging tables shown on the far right of the diagram. Staff then have the ability to review this pending patron information and approve account creation.


===== User stories for BCSC integration =====

  - As a new patron, I want to register for a library card online so that I can access library resources more easily.
  - As circulation staff, I want new patrons to pre-register for an account online so that the registration process is more efficient and accessible and patron information is entered quickly and reliably.
  - As circulation staff, I want to auto-validate current patrons when I renew their accounts so that I don't have to do it manually at the circ desk.
  - As a library manager, I want patron accounts to be pre-registered/validated to reduce workload at the circ desk and improve reliability of patron information.
  - As a Co-op service manager, I want patron data to be validated so that I can use it as a basis for other services.
  - As a malicious user, I want to create fake or unauthorized patron accounts so that I can access library resources illegitimately.


===== Creating a patron self-registration application =====

Suppose we wanted to create an application that allowed patrons to request a library card using their BC Services Card. That is, instead of using EG's native patron self-registration UI, we'd have an external app that presents the user with a registration form, populates with their personal data based on their BC Services Card, and then submits that data to Evergreen to create a "pending" patron account (which would then be reviewed and approved by library staff within Evergreen, just like native self-registration).

How does our application submit the patron data to Evergreen? Our best bet is to use the existing infrastructure for native patron self-registration as much as possible -- that is, create pending patron accounts in the same way that Evergreen itself does.

Native self-reg uses the ''open-ils.actor.user.stage.create'' API. This endpoint accepts the following parameters:
  - user (stgu object)
  - mailing address (stgma object)
  - billing address (stgma object)
  - stat cats (array of stgsc objects)
  - user settings (array of stgs objects)

Only the first parameter (user) is required, the others are optional. The object classes (stgu, stgma, stgsc, stgs) are defined by the [[https://docs.evergreen-ils.org/eg/docs/latest/development/intro_opensrf.html#_accepting_and_returning_evergreen_objects|fieldmapper]].

How do we use this API? There are a few possibilities:

1. We could use it directly, by submitting HTTP requests to the [[https://wiki.evergreen-ils.org/doku.php?id=osrfhttp:opensrf_gateway|OpenSRF gateway]], like so:

<code>https://<domain>/osrf-gateway-v1?service=open-ils.actor&method=open-ils.actor.user.stage.create&param=[<user object>]</code> 

For this to work, our application would need to know how to instantiate Evergreen-specific fieldmapper objects (stgu, etc.) that will be accepted as such when passed to EG via the gateway, which is non-trivial. This is a brittle approach for an external application.

2. We could build BC Services Card integration directly into Evergreen. This is probably not a good idea, since it requires developers to be knowledgeable about both Evergreen and BC Services Card. Also, we want to leave the door open to using our hypothetical patron self-registration application with non-Evergreen library systems.

3. We could create a thin wrapper service in Evergreen that accepts patron data as a JSON blob, translates it into native Evergreen objects, and passes the result to the ''open-ils.actor.user.stage.create'' API. This would mean that our application doesn't have to know anything about Evergreen objects or the fieldmapper. The service could also include some kind of simple "API key" type mechanism that indicates which application provided the patron data; this could be leveraged to speed up the approval process, since BC Services Card-based account requests are presumed to be trustworthy. For simplicity's sake, the service could be exposed as another Evergreen API via the OpenSRF gateway. This would obviously require some Evergreen-specific development, but implementation would not be an enormous project for a knowledgeable Evergreen developer, and the actual self-registration application could be built separately by devs who don't need to have much experience with Evergreen.

4. We could skip the API and allow the application to insert patron data directly into staging.user_stage and related tables in Evergreen's Postgres database.

==== Matching existing accounts ====

The staff interface for patron registration checks for duplicates during the registration process: it will present a notification if the provided name, email, or address already exists in Evergreen. The duplicate checks are performed using the ''open-ils.actor.patron.search.advanced'' API, which will return the IDs of any patron accounts where the fields you specify (e.g. email) are an exact match for the values you specify (e.g. "foo@example.com"). This API requires an auth token, which is basically the session token you receive upon authenticating. It is possible to obtain an auth token via the OpenSRF gateway ([[https://wiki.evergreen-ils.org/doku.php?id=newdevs:apis|see here]]), but you will need login credentials for a user that has permission to retrieve patron data; only patrons that are accessible to that user will be included in search results.

Many BC public library patrons have their driver's license or BC Services Card number in the ident_value field, but this data is not controlled or validated and we can't guarantee its accuracy.

==== Further information ====


[[https://docs.evergreen-ils.org/eg/docs/latest/integrations/web_services.html|More information about Evergreen web services (OpenSRF gateway etc.)]]. Some of this info may be outdated.

[[https://docs.evergreen-ils.org/eg/docs/latest/development/intro_opensrf.html|Easing Gently into OpenSRF]] - an overview of how to write an OpenSRF service; provides a good introduction to some OpenSRF concepts and how to use them.

[[https://wiki.evergreen-ils.org/doku.php?id=newdevs:start|Documentation for new Evergreen developers]] - very incomplete, but perhaps useful for orientation purposes.

----
===== Coop Answers to Questions from March 16 Kickoff Call doc =====

  * **Inventory of current technologies being used**
    * Evergreen uses an open source software stack, primarily Linux + Apache + PostgreSQL + Perl. The public catalogue (OPAC) uses Perl's Template Toolkit engine with some Bootstrap JS. The staff client is an Angular app that makes use of Websockets. [[https://wiki.evergreen-ils.org/doku.php?id=eg_developer_overview|Additional details here.]]
  * **List of current library sites and what is being managed for them**
  * **Any additional common challenges today not identified above**
  * **General intro to the library system, what changes and what’s static,**
    * Not really sure how to answer this, can you be more specific about what you want to know?
  * **How are people and accounts created today? Go to branch -> some registration and card issuance, create account at home. What identifiers are collected, email, names, ?**
    * Potential workflows:
      - A new public library patron wants a library account. They go into their local library, go to the circ desk, and show their ID or proof of residency. Circ staff manually enter their info into the patron registration UI (city and province are auto-populated when they enter the postal code) and give the patron a physical library card with barcode.
      - A new public library patron wants a library account. They pre-register for an account online. Circ staff review, approve the account, and send a notification to the patron. The patron goes into their local library and picks up their library card at the circ desk. Note that this workflow still involves manual intervention by staff for each new account registration. <del>Sitka is not currently using this feature, but it is supported by Evergreen.</del>  A dozen or so Sitka libraries are currently using this feature to allow patrons to "Request a Library Card."
      - An existing patron's account expires. They go into their local library to renew their account, or they discover their account is expired when they're already at the circ desk. Circ staff check their ID to verify their current personal information, and update the account if needed. NB: Some libraries may permit renewal over the phone rather than in-person, but staff approval is definitely required. Libraries may email patrons prior to account expiry as a form of customer engagement.
    * The above is based on Jeff's educated assumptions about how patron registration works. We should consult with a few of our libraries to verify the above and see if there's anything else we should take into account. We should also talk to non-Sitka libraries that are currently using ID verification services about how it fits into their workflows.
    * During registration, the patron is assigned to a patron group/type/profile which determines their access to services. Current groups are:
      * PL Adult (default)
      * PL Juvenile
      * PL Teen
      * PL BC OneCard
      * PL Non Resident - Adult
      * PL Non Resident - Juvenile
      * PL Federation
      * PL Extended Loans
      * PL No-fines
      * PL Print Disabled
      * PL Home Services
      * PL Restricted Access
      * PL Custom
      * PL Temporary
      * PL New User
    * Identifiers collected (plus % of current BC public library patrons who have this info in their patron record):
      * legal name: 100%
      * preferred name: 2%
      * address: 99%
      * phone numbers: 100%
      * email: 90%
      * date of birth: 49%
      * ID number:
        * driver's license: 11%
        * BC ID/BC Services Card: 1%
        * Status Card: <1%
        * Other (unknown type): 5%
    * Examples of other data gathered during patron registration for reporting purposes (a.k.a. statistical categories):
      * geographic area, residency, electoral district
      * gender (mostly unused now)
      * school, grade
      * age group
  * **When we say “record” what do we mean?**
    * A "patron record" is an account which exists as a row in the actor.usr table. This account is linked to the patron's circulations, holds, staff notes, user messages/notifications, fines, and other transactions or user activity recorded in the database.
  * **How strict are validation rules? How strict would they be with bcsc?**
    * We don't do any automated validation of personal info. Personal information is entered manually by staff (except city/province, which is usually auto-populated from the postal code) and verified manually when accounts are renewed. Email and phone number are used for contacting patrons (e.g. password reset requires a valid email address) and may also be updated by the patron by logging into the catalogue.
  * **Is there business value for existing patrons to “link” their BCSC proactively? Or is this mainly only for password resets, new signups/account registration etc?**
  * **Privacy/security requirements and oversight - what are the Co-Ops current approaches?**
      * **IDENTOS can help identify how this aligns to BCSC onboarding requirements**
  * **Conversation with non-SITKA libraries that would benefit from having access to brokering of BCSC? What are their requirements that differ from Co-Op?**
  * **Is there a desire to have additional / complementary digital library access vs overdrive?**
     * Coop Response: Not sure I fully understand; the Coop (and non-Sitka member libraries) provide a TON of access to digitally licensed products right now. Those 3rd party licensed content providers typically authenticate against accounts from the library ILS through a couple of different methods. The Co-op offers SIP2 interfaces (which were historically developed for in-branch authentication only but have for 20 years been mis-used for cross-internet authentication) as well as an emulation of the "PatronAPI" auth interface, the closest to any sort of industry de facto standard that has emerged that at least looks like a recognizable and securable web pattern. Other libraries offer ezproxy as another option. Longer term we would love to see both libraries and the publishing industry they interact with move towards more 21st century methods like OAuth, but there is a chicken/egg issue there. The only connection I see between this project and that is the extent to which we get some experience as a SAML identity consumer, maybe we get some chops for a day when we need to be an IdP, but right now these are not otherwise directly connected issues. Apologies if I misunderstood the question.
  * **Any other tables and key data elements that would be helpful (other than those already provided on the wiki) ?**

==== Further questions from Co-op staff ====

  * There may be information that needs to be captured during registration that can't be validated by the service or provided by the patron (e.g. patron type, stat cats). How to handle that?
  * Evergreen workflows require staff to manually approve pre-registered accounts and provide a physical card. Is there a desire to enable account creation without these steps? 
  * Should we be worried about fraudulent account creation? For example, theoretically someone could register an account with stolen info and steal a bunch of books, and the victim could be on the hook for replacement costs. This becomes a bigger risk if we become the ID provider for other services.
  * Are there other privacy/security risks to be considered?

===== Docs from Other ILS =====
While our initial focus is on validating potential Sitka patrons via the BC Services Card, actually making this work will require standing up some sort of SAML server on our end and liasing with the BC Services Card to integrate. There are approx 20 other BC libraries using ±4 other ILS who, if they also wanted to offer validation via the Services Card, would need to standup similar SAML servers. So a secondary goal of this project is to see if we can architect our solution in such a way as it is the sector-wide integration against the Services Card, with us doing integration with the libraries. To that end, I asked other libraries (via the BC Libraries and IT list) to share any API docs they might have that can inform the consultants in architecting a solution. Below is what resulted:


^ Non-Evergreen ILS in BC ^ Library Name  ^ Expressed Interest ^
|{{ :public:polarisapireferenceguide_7.3.pdf |Innovative Polaris}} | Coquitlam Public Library	|x |
| |Okanagan Regional Library| |
|[[https://techdocs.iii.com/sierraapi/Content/zAPIs/patronAPI.htm|Innovative Sierra]] | Fraser Valley Regional Library	| x|
| |Richmond Public Library | x|
| |Thompson-Nicola Regional District Library| |
| |Vancouver Island Regional Public Library | |
| |West Vancouver Memorial Library| |
|{{ :public:web_services_for_horizon_6.2.2_setup_guide.pdf |SirsiDynix Horizon}} |Burnaby Public Library |x |
| |Cranbrook Public Library | |
| |New Westminster Public Library| |
| |North Vancouver District Library	| |
| |Penticton Public Library| |
| |Port Moody Public Library	| |
| |Surrey Public Library |x |
| |Vancouver Public Library | x|
|SirsiDynix Symphony|Greater Victoria Public Library |x |
| |North Vancouver City Library | |
| |Powell River Public Library	| |
| |Prince George Public Library	| |
User Tools

Page Tools

BC Libraries Coop wiki

Site Tools
