No vendor uses plain SQL as the primary interface for terminal users — they all have proprietary query languages optimized for financial workflows. Behind the scenes, they may use SQL databases, Hadoop, or columnar stores (e.g., kdb+, Arctic, Parquet) internally for storage and processing, then expose data through their own APIs.

Bloomberg is the most “closed” — almost everything stays inside the Bloomberg Terminal. FactSet and Refinitiv are more open with data export.

How Do They Collect and Organize the Data?

Collection (Data Explorers / Securities Lending)

Securities lending data specifically comes from:

• Agent lenders (custodian banks like State Street, BNY Mellon, JPMorgan) report their lending books
• Prime brokers report borrow/loan positions
• Beneficial owners (pension funds, mutual funds) report what they lend out

The data aggregator (originally Data Explorers, now S&P Global Market Intelligence) collects this via:

• Daily automated feeds (SFTP/API) from ~20,000+ institutional participants
• Standardization — normalizing different formats into a common schema
• Anonymization — individual lender/borrower identities are masked; only aggregates are published

Organization

The data is typically organized along these dimensions:

• Security (ISIN, SEDOL, CUSIP)
• Date (daily snapshots)
• Metrics — e.g., BO_INVENTORY_QUANTITY (shares available to borrow), utilization, fees, on-loan value
• Adjustments — corporate action adjusted vs. raw

FactSet then ingests this into their .FDB format and makes it queryable through their scripting language.

Each company have their own proprietary language to store data in binary and process super fast, for example

double_array DX_SUPPLY(fds_date sdate=0, fds_date edate=sdate, string frq="Daily",
string curn="LOCAL", string option1="", string option2="")
{
double_array result;
curn    = UPPERCASE(curn);
option1 = UPPERCASE(option1);
option2 = UPPERCASE(option2);
if (option1 == "QUANTITY") {
  if (UPPERCASE(option2) == "TOTAL")
    result = FDB_DOUBLE_FB("DATAEXP_DB:DATA_EXPLORERS.FDB", "BO_INVENTORY_"+option1,           sdate,edate,frq,,"ADJ");
  else
    result = FDB_DOUBLE_FB("DATAEXP_DB:DATA_EXPLORERS.FDB", "BO_INVENTORY_"+option1+"_"+option2, sdate,edate,frq,,"ADJ");
}
else if (curn == "LOCAL" || curn == "") {
  if (UPPERCASE(option2) == "TOTAL")
    result = FDB_DOUBLE_FB(..., "BO_INVENTORY_"+option1,            ...) * EXRATE(DX_CURRENCY(...), PISO(), PDNC(...));
  else
    result = FDB_DOUBLE_FB(..., "BO_INVENTORY_"+option1+"_"+option2, ...) * EXRATE(DX_CURRENCY(...), PISO(), PDNC(...));
    result = FDB_DOUBLE_FB("DATAEXP_DB:DATA_EXPLORERS.FDB", "BO_INVENTORY_"+option1+"_"+option2, sdate,edate,frq,,"ADJ");
}
else if (curn == "LOCAL" || curn == "") {
  if (UPPERCASE(option2) == "TOTAL")
    result = FDB_DOUBLE_FB(..., "BO_INVENTORY_"+option1,            ...) * EXRATE(DX_CURRENCY(...), PISO(), PDNC(...));
  else
    result = FDB_DOUBLE_FB(..., "BO_INVENTORY_"+option1+"_"+option2, ...) * EXRATE(DX_CURRENCY(...), PISO(), PDNC(...));
}
else {
  // explicit currency
  if (UPPERCASE(option2) == "TOTAL")
    result = FDB_DOUBLE_FB(...) * EXRATE(DX_CURRENCY(...), curn, PDNC(...));
  else
    result = FDB_DOUBLE_FB(...) * EXRATE(DX_CURRENCY(...), curn, PDNC(...));
}
return result;