Cobalt Speech: API Documentation – Privacy Screen

Docs: Server Setup

Mon, 01 Jan 0001 00:00:00 +0000

Installing Cobalt Privacy Screen

Cobalt distributes a docker-compose file that orchestrates three docker images, one for each of the following services:

Privacy Screen Server (frontend for accepting text / audio streams)
Transcribe Server (for recognizing text in audio files)
Redaction Backend Engine (for redacting text data)

Having these components as separate images facilitates large deployments where each image can be auto-scaled independently based on request traffic.

Installing Server

Contact Cobalt to get a link to the image files in AWS S3 and the docker-compose configuration file. This link will expire in two weeks, so be sure to download the file to your own server.

Download with the AWS CLI if you have it, or with curl:

URL="the url sent by Cobalt"
FILE_NAME="name you want to give the file (should end with the same extension as the url, usually tar.bz2)"
curl $URL -L -o $FILE_NAME

Untar the file, and load the docker images. The tar file will also contain the docker-compose.yaml file.
```
tar -xvjf $FILE_NAME -C ./
docker load < *.bz2
```
Copy the cobalt license file into the server folder
Copy the deid license file into the server folder
Start the services using docker-compose:
```
docker-compose up --build
```

The server will be running in the container and listening on port 2728 for gRPC requests from clients.

Docs: Connecting to the Server

Mon, 01 Jan 0001 00:00:00 +0000

Once you have the Cobalt Privacy Screen server up and running, you are ready to create a client connection.

First, you need to know the address (host:port) where the server is running. This document will assume the values 127.0.0.1:9002, but these can be replaced with your server address in actual code.

Default Connection

The following code snippet connects to the server and queries its version. It uses our recommended default setup, expecting the server to be listening on a TLS encrypted connection.

package main

import (
	"context"
	"fmt"
	"log"

	"github.com/cobaltspeech/sdk-trifid/grpc/go-trifid"
)

const serverAddr = "127.0.0.1:9002"

func main() {
	client, err := trifid.NewClient(serverAddr)
	if err != nil {
		log.Fatal(err)
	}

	// Be sure to close the client when we are done with it.
	defer client.Close()
}

import trifid

client = trifid.Client(server_address="localhost:9002")

Insecure Connection

It is sometimes required to connect to Privacy Screen server without TLS enabled (during debugging, for example). Note that if the server has TLS enabled, attempting to connect with an insecure client will fail.

To create an insecure connection, do the following when creating the client:

client, err := trifid.NewClient(serverAddr, trifid.WithInsecure())

client = trifid.Client(server_address="localhost:9002", insecure=True)

Client Authentication

In our recommended default setup, TLS is enabled in the gRPC setup, and when connecting to the server, clients validate the server’s SSL certificate to make sure they are talking to the right party. This is similar to how “https” connections work in web browsers.

In some setups, it may be desired that the server should also validate clients connecting to it and only respond to the ones it can verify. If your Privacy Screen server is configured to do client authentication, you will need to present the appropriate certificate and key when connecting to it.

Please note that in the client-authentication mode, the client will still also verify the server’s certificate, and therefore this setup uses mutually authenticated TLS. This can be done with:

// certPem and keyPem are the bytes of the client certificate and key
// provided to you.
client, err := trifid.NewClient(serverAddr, trifid.WithClientCert(certPem, keyPem))

# cert_pem and key_pem are the contents of the client certificate and key
# provided to you.
client = trifid.Client(server_address="localhost:9002", client_certificate=cert_pem client_key=key_pem)

Server Information

The client provides two methods to get information about the server - Version and ListModels.

Version

The Version method provides information about the version of the Privacy Screen server the client is connected to, as well as information about other relevant services and packages the server uses.

// Request the server version info
ver, err := client.Version(context.Background())
fmt.Printf("Server Version: %v\n", ver)

# Request the server version info
ver = client.version()
print(f"Server Version: {ver}")

List Models

The ListModels method fetches a list of models available on the Privacy Screen server. On the server side, the models are specified as part of the server’s config file.

// Request the list of models
modelList, err := client.ListModels(context.Background())
fmt.Printf("Available Models:\n")
for _, mdl := range modelList.Models {
	fmt.Printf("  ID: %v\n", mdl.Id)
	fmt.Printf("    Name: %v\n", mdl.Name)
	fmt.Printf("    Redaction Classes: %v\n", mdl.RedactionClasses)
}

# Request the list of models
model_list = client.list_models()
print("Available Models:")
for mdl in model_list:
    print(f"  ID: {mdl.id}")
    print(f"    Name: {mdl.name}")
    print(f"    Redaction Classes: {mdl.redaction_classes}")

Docs: Text Redaction

Mon, 01 Jan 0001 00:00:00 +0000

TODO

Docs: Concurrency

Mon, 01 Jan 0001 00:00:00 +0000

The recommended level of concurrency, i.e. the optimal number of simultaneous requests to make to the container is covered below for the CPU and GPU containers. The recommended concurrency level is driven primarily by the compute requirement of the Neural Network models, such as for PII detection.

CPU

For Neural Network inference workloads, CPUs don’t require inputs to be batched together to achieve good hardware utilization. In practice, due to network overhead and pre/post-processing code it is best to use a low level of concurrency such as 2 per container instance. If latency isn’t a concern, a value of 32 is recommended.

GPU

Unlike CPUs, GPUs require inputs to be batched together and processed as a single large input to achieve optimal hardware utilization. This means that there is a tradeoff between latency and throughput. A concurrency level of 32 per container instance is a good tradeoff between latency and throughput, however concurrency levels as low as 8 do not significantly impact throughput. If latency isn’t a concern, a value of 128 will ensure maximum hardware utilization.

Docs: Privacy Screen Client

Mon, 01 Jan 0001 00:00:00 +0000

This release includes a PrivacyScreen client that can be used to quickly send audio/transcripts to the server. This reads audio files in WAV (PCM16SLE) format and transcripts in the JSON format, you can see examples for txt and json files:

Jack and Jill went up to 224 North Hill drive to fetch a pail of water. Jack fell down broke his crown and Jill called 4125555555.

{
    "utterances": [
        {
            "start_time_ms": 30,
            "duration_ms": 4230,
            "audio_channel": 0,
            "words": [
                {
                    "start_time_ms": 30,
                    "duration_ms": 390,
                    "text": "Jack"
                },
                {
                    "start_time_ms": 420,
                    "duration_ms": 120,
                    "text": "and"
                },
                {
                    "start_time_ms": 540,
                    "duration_ms": 240,
                    "text": "Jill"
                },
                {
                    "start_time_ms": 780,
                    "duration_ms": 240,
                    "text": "went"
                },
                {
                    "start_time_ms": 1020,
                    "duration_ms": 150,
                    "text": "up"
                },
                {
                    "start_time_ms": 1170,
                    "duration_ms": 60,
                    "text": "to"
                },
                {
                    "start_time_ms": 1230,
                    "duration_ms": 1080,
                    "text": "224"
                },
                {
                    "start_time_ms": 2310,
                    "duration_ms": 300,
                    "text": "North"
                },
                {
                    "start_time_ms": 2610,
                    "duration_ms": 150,
                    "text": "Hill"
                },
                {
                    "start_time_ms": 2760,
                    "duration_ms": 300,
                    "text": "drive"
                },
                {
                    "start_time_ms": 3060,
                    "duration_ms": 90,
                    "text": "to"
                },
                {
                    "start_time_ms": 3150,
                    "duration_ms": 270,
                    "text": "fetch"
                },
                {
                    "start_time_ms": 3420,
                    "duration_ms": 60,
                    "text": "a"
                },
                {
                    "start_time_ms": 3480,
                    "duration_ms": 270,
                    "text": "pail"
                },
                {
                    "start_time_ms": 3750,
                    "duration_ms": 120,
                    "text": "of"
                },
                {
                    "start_time_ms": 3870,
                    "duration_ms": 390,
                    "text": "water."
                }
            ]
        },
        {
            "start_time_ms": 9300,
            "duration_ms": 5324,
            "audio_channel": 1,
            "words": [
                {
                    "start_time_ms": 9300,
                    "duration_ms": 420,
                    "text": "Jack"
                },
                {
                    "start_time_ms": 9720,
                    "duration_ms": 210,
                    "text": "fell"
                },
                {
                    "start_time_ms": 9930,
                    "duration_ms": 420,
                    "text": "down"
                },
                {
                    "start_time_ms": 10410,
                    "duration_ms": 270,
                    "text": "broke"
                },
                {
                    "start_time_ms": 10680,
                    "duration_ms": 150,
                    "text": "his"
                },
                {
                    "start_time_ms": 10830,
                    "duration_ms": 450,
                    "text": "crown"
                },
                {
                    "start_time_ms": 11310,
                    "duration_ms": 180,
                    "text": "and"
                },
                {
                    "start_time_ms": 11490,
                    "duration_ms": 210,
                    "text": "Jill"
                },
                {
                    "start_time_ms": 11700,
                    "duration_ms": 330,
                    "text": "called"
                },
                {
                    "start_time_ms": 12030,
                    "duration_ms": 2594,
                    "text": "4125555555."
                }
            ]
        }
    ]
}

Examples of client calls

There are several ways the client interacts with the server. These examples are always run from the same path as the client binary is. When in doubt, run ./privacy-screen-gprc-client -h to get more information the parameters needed to run the client.

Redact Text

./privacy-screen-grpc-client redact-text \
        --insecure \
        --model-id general \
        --input-text input.txt \
        --output-result redacted_token.json

Redact Transcript

./privacy-screen-grpc-client redact-transcript \
        --insecure \
        --model-id general \
        --input-transcript testdata/input.json \
        --output-transcript redacted_output.json

Redact Transcribed Audio

./privacy-screen-grpc-client redact-transcribed-audio \
        --insecure \
        --model-id general \
        --input-audio testdata/input.wav \
        --input-transcript testdata/input.json \
        --output-audio redacted_output.wav \
        --output-transcript redacted_output.json \
        --timeout 5m

Transcribe and Redact

./privacy-screen-grpc-client transcribe-and-redact \
        --insecure \
        --model-id en_US \
        --input-audio testdata/input.wav \
        --output-audio redacted_output.wav \
        --output-transcript redacted_output.json \
        --output-unredacted-transcript unredacted_output.json \
        --timeout 5m

Docs: Redaction Categories

Mon, 01 Jan 0001 00:00:00 +0000

Personally Identifiable Information (PII)

Label	Description	Regulatory Compliance
ACCOUNT_NUMBER	Customer account or membership identification number Policy No. 10042992; Member ID: HZ-5235-001 Note: Full support for English; Multilingual support in progress	HIPAA_SAFE_HARBOR, CCI
AGE	Numbers associated with an individual’s age 27 years old; 18 months old More details When given in years, only the number is flagged, but both number and time unit are flagged when given in other units like months or weeks Also includes age ranges: 29-35 years old; 18+; A man in his forties	GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
DATE	Specific calendar dates, which can include days of the week, dates, months, or years Friday, Dec. 18, 2002; Dated: 02/03/97 See also: DATE_INTERVAL, DOB More details If no calendar date is specified, days of the week are not flagged: Your appointment is on Monday Indexical terms are not flagged: yesterday; tomorrow	HIPAA_SAFE_HARBOR, Quebec Privacy Act, CCI
DATE_INTERVAL	Broader time periods, including date ranges, months, seasons, years, and decades 2020-2021; 5-9 May; January 1984 See also: DATE, DOB	HIPAA_SAFE_HARBOR, CCI
DOB	Dates of birth Born: March 7, 1961 See also: DATE, DATE_INTERVAL	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
DRIVER_LICENSE	Driver's permit numbers DL# 134711-320 See also: VEHICLE_ID More details Includes International Driving Permits (IDP) and Pilot’s licenses	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
DURATION	Periods of time, specified as a number and a unit of time 8 months; 2 years Note: Full support for English; Multilingual support in progress
EMAIL_ADDRESS	Email addresses info@cobaltspeech.com	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI, CCI
EVENT	Names of events or holidays Olympics; Yom Kippur
FILENAME	Names of computer files, including the extension or filepath Taxes/2012/brad-tax-returns.pdf	CCI
GENDER	Terms indicating gender identity, including slang terms. Note that performance is stronger for terms that are more likely to occur in formal documents, such as "male", "transgender", "non-binary", "female", "M", "F", etc. Other terms, such as "woman", "gentleman", etc., may not be captured in every context. female; trans	CPRA, GDPR, GDPR Sensitive, APPI Sensitive
HEALTHCARE_NUMBER	Healthcare numbers and health plan beneficiary numbers Policy No.: 5584-486-674-YM More details Includes medical record numbers, health insurance policy/account numbers, and member IDs, for example, German Sozialversicherungsnummer (also used as SSN), Philippine PhilHealth ID number, Ukrainian VHI number	CPRA, GDPR, HIPAA, Quebec Privacy Act, APPI
IP_ADDRESS	Internet IP address, including IPv4 and IPv6 formats 192.168.0.1 2001:db8:0:0:0:8a2e::7334	CPRA, GDPR, HIPAA, Quebec Privacy Act, APPI
LANGUAGE	Names of natural languages Korean; French	GDPR, GDPR Sensitive, APPI Sensitive
LOCATION	Metaclass for any named location reference; See subclasses below Eritrea; Lake Victoria More details May co-occur with ORGANIZATION when the context refers explicitly to the organization’s location The patient was transferred to Northwest General Hospital	GDPR, HIPAA_SAFE_HARBOR, APPI, CCI
LOCATION_ADDRESS	Full or partial physical mailing addresses, which can include: building name or number, street, city, county, state, country, zip code 25/300 Adelaide T., Perth WA 6000, Aus. 145 Windsor St. Mail to: Kollwitzstr 13, 10405, Berlin	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI, CCI
LOCATION_ADDRESS_STREET	A subclass of `LOCATION_ADDRESS`, covering: a building number and street name, plus information like a unit numbers, office numbers, floor numbers and building names, where applicable 25/300 Adelaide T., Perth WA 6000, Aus. 145 Windsor St. Mail to: Kollwitzstr 13, 10405, Berlin	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI, CCI
LOCATION_CITY	Municipality names, including villages, towns, and cities Toronto; Berlin; Denpasar	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI, CCI
LOCATION_COORDINATE	Geographic positions referred to using latitude, longitude, and/or elevation coordinates We’re at 40.748440 and -73.984559	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
LOCATION_COUNTRY	Country names Canada; Namibia	GDPR, APPI, CCI
LOCATION_STATE	State, province, territory, or prefecture names Ontario; Arkansas; Ich lebe in NRW	GDPR, APPI, CCI
LOCATION_ZIP	Zip codes (including Zip+4), postcodes, or postal codes 90210; B2N 3E3 More details Optimized for various English-speaking locales (Australia, Canada, United Kingdom, United States), as well as international equivalents	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI, CCI
MARITAL_STATUS	Terms indicating marital status single; common-law; ex-wife; married	APPI Sensitive
MONEY	Names and/or amounts of currency 15 pesos; $94.50	CCI
NAME	Names of individuals, not including personal titles such as ‘Mrs.’ or ‘Mr.’ Dwayne Johnson; Mr. Khanna	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
NAME_FAMILY	Names indicating a person’s family or community; often a last name in Western cultures and first name in Eastern cultures François Truffaut; Ozu Yasujirō	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
NAME_GIVEN	Names given to an individual, usually at birth; often first / middle names in Western cultures and middle / last names in Eastern cultures François Truffaut; Ozu Yasujirō	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
NAME_MEDICAL_ PROFESSIONAL	Full names, including professional titles and certifications, of medical professional, such as doctors and nurses Attending physician: Dr. Kay Martinez, MD	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
NUMERICAL_PII	Numerical PII (including alphanumeric strings) that doesn't fall under other categories. See also a section below on international variants as some of them are mapped to this category, for example, Belgian BTW nummer or European VAT number. More details Includes the following: numbers in the medical field, such as device serial numbers, POS codes, NPI numbers, etc.; computer numbers like MAC addresses, cookie IDs, VPNs, error codes, access codes, message IDs, etc.; business-related numbers like DUNS numbers, company registration numbers, provider IDs, etc.; numbers related to purchasing, like order IDs, transaction numbers, confirmation numbers, tracking numbers, etc.; also numbers assigned to various forms of IDs, files, documents, proceedings, invoices, claim IDs, record IDs, etc.	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI, CCI
OCCUPATION	Job titles or professions professor; actors; engineer; CPA	Quebec Privacy Act, APPI, CCI
ORGANIZATION	Names of organizations or departments within an organization BHP; McDonald's; LAPD More details May co-occur with LOCATION when the context refers explicitly to the organization’s location Donations can be brought to Royal Canadian Legion Branch 43	Quebec Privacy Act, APPI, CCI
ORGANIZATION_MEDICAL_ FACILITY	Names of medical facilities, such as hospitals, clinics, pharmacies, etc. Northwest General Hospital; Union Family Health Clinic	Quebec Privacy Act, APPI
ORIGIN	Terms indicating nationality, ethnicity, or provenance Canadian; Sri Lankan	CPRA, GDPR, GDPR Sensitive, Quebec Privacy Act, APPI Sensitive
PASSPORT_NUMBER	Passport numbers, issued by any country PA4568332; NU3C6L86S12	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
PASSWORD	Account passwords, PINs, access keys, or verification answers 27%alfalfa; temp1234 My mother's maiden name is Smith	CPRA, APPI, CCI
PHONE_NUMBER	Telephone or fax numbers +4917643476050	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
PHYSICAL_ATTRIBUTE	Distinctive bodily attributes, including terms indicating race I'm 190cm tall; He belongs to the Black students’ association	CPRA, GDPR, GDPR Sensitive, APPI Sensitive
POLITICAL_AFFILIATION	Terms referring to a political party, movement, or ideology liberal; Republican	CPRA, GDPR, GDPR Sensitive, Quebec Privacy Act, APPI Sensitive
RELIGION	Terms indicating religious affiliation Hindu; Presbyterian	CPRA, GDPR, GDPR Sensitive, Quebec Privacy Act, APPI Sensitive
SEXUALITY	Terms indicating sexual orientation, including slang terms bisexual; gay; straight	CPRA, GDPR, GDPR Sensitive, APPI Sensitive
SSN	Social Security Numbers or international equivalent government identification numbers 078-05-1120; *--3256* More details Includes, for example, Australian TFN, Belgian NISS, British NIN, Canadian SIN, Dutch BSN, German Sozialversicherungsnummer (also used as a healthcare number, see: HEALTHCARE_NUMBER), French INSEE, Indian Aadhaar, Italian TIN, Philippine SSS, Spanish NUSS, Ukrainian TIN, and Mexican NSS formats. Flags mentions of complete numbers as well as the last four digits only.	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
TIME	Expressions indicating clock times 19:37:28; 10pm EST	CCI
URL	Internet addresses www.cobaltspeech.com	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, CCI
USERNAME	Usernames, login names, or handles cobaltspeechandlanguage; @_CobaltSpeechAndLanguage	CPRA, GDPR, APPI
VEHICLE_ID	Vehicle identification numbers (VINs), vehicle serial numbers, and license plate numbers 5FNRL38918B111818; BIF7547 See also: DRIVER_LICENSE	CPRA, GDPR, HIPAA_SAFE_HARBOR, APPI, CCI
ZODIAC_SIGN	Names of Zodiac signs Aries; Taurus

Protected Health Information (PHI)

Label	Description	Regulatory Compliance
BLOOD_TYPE	Blood types She's type AB positive	CPRA, GDPR, Quebec Privacy Act
CONDITION	Names of medical conditions, diseases, syndromes, deficits, disorders chronic fatigue syndrome; arrhythmia; depression	CPRA, GDPR, Quebec Privacy Act, APPI Sensitive
DOSE	Medically prescribed quantity of a medication limit intake to 700 mg/day
DRUG	Medications, vitamins, and supplements advil; Acetaminophen; Panadol	CPRA, GDPR, Quebec Privacy Act, APPI Sensitive, CCI
INJURY	Bodily injuries, including mutations, miscarriages, and dislocations I broke my arm; I have a sprained wrist	CPRA, GDPR, Quebec Privacy Act, APPI Sensitive
MEDICAL_PROCESS	Medical processes, including treatments, procedures, and tests heart surgery; CT scan	CPRA, GDPR, Quebec Privacy Act, APPI Sensitive, CCI
STATISTICS	Medical statistics 18% of patients	Quebec Privacy Act

Payment Credit Industry Information (PCI)

Label	Description	Policy & Regulatory Compliance
BANK_ACCOUNT	Bank account numbers and international equivalents, such as IBAN Acct. No.: 012345-67	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI, CCI
CREDIT_CARD	Credit card numbers 0123 0123 0123 0123 ** **4252 More details Includes debit, ATM, Direct Debit, PrePay, Charge Cards, and support for cards that do not have 16 digits such as American Express or China UnionPay cards. Flags mentions of complete numbers as well as the last four digits only.	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI, CCI
CREDIT_CARD_ EXPIRATION	Expiration date of a credit card Expires: July 2023; Exp: 02/28	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI
CVV	3- or 4-digit card verification codes and equivalents CVV: 080 More details Includes institution-specific variants: American Express: CID (card ID), CVD (card verification data) CSC / 3CSC (card security code) China UnionPay: CVN (card validation number) CIBC Mastercard: SPC (signature panel code) Discover: CID (card ID), CVD (card verification data) ELO (Brazil): CVE (Elo verification code) JCB (Japan Credit Bureau): CAV (card authentication value) Mastercard: CVC (card validation code) VISA: CVV (card verification value)	CPRA, GDPR, HIPAA_SAFE_HARBOR, Quebec Privacy Act, APPI, CCI
ROUTING_NUMBER	Routing number associated with a bank or financial institution 012345678 More details Includes international equivalents: Canadian & British sort codes, Australian BSB numbers, Indian Financial System Codes, Branch/transit numbers, Institution numbers, and Swift codes	CCI

Beta Entity Types

Note that Beta support for the following entity types is currently only available with our English models.

Label	Description	Regulatory Compliance
CORPORATE_ACTION	Any action a company takes that could affect its stock value or its shareholders Bridge Investment Group LLC (later renamed Bridge Investment Group Holdings LLC); We’ve merged two neighboring retail locations	CCI
FINANCIAL_METRIC	Financial metrics or financial ratios are quantitative indicators of a company’s financial health adjusted earnings per share declined year-over-year; Online sales slow as UK shoppers rein in Christmas spending	CCI
MEDICAL_CODE	Codes belonging to medical classification systems such as SNOMED, ICD-10, NDC, etc. 1981-03-11T04:11:32-03:00 Forearm sprain SNOMED-CT 70704007; <medcode type="string"> R74.8 <desc type="string">Abnormal levels of other serum enzymes	CPRA, GDPR, GDPR Sensitive, Quebec Privacy Act, APPI Sensitive
PRODUCT	Names or model numbers of items made by an organization; includes intangible products like software and games, as well as other services iPhone; Toyota Camry	CCI
TREND	A description of the “quality” or the direction in which a financial measurement is going reflecting the accelerating shift of off-line to online; amid rising costs and shrinking profits	CCI

Each of the beta entities must be enabled explicitly in the deid request. This can be done by setting the redaction class flag in the client: ./privacy-screen-grpc-client redact-text --insecure --model-id general --input-text input.txt --output-result redacted_token.json --redaction-classes="AGE,BANK_ACCOUNT"

International Entity Mapping

In the tables below, you can find localized variants of our entity types. For each entity type, there is a description, an example, and the label under which the entity falls. This section does not include entity types that may vary regionally, but still directly correspond to one of the entities listed above (e.g., PHONE_NUMBER, PASSPORT_NUMBER, DRIVER_LICENSE, LOCATION_ADDRESS, CREDIT_CARD_EXPIRATION). The following numbers are commonly used across many countries, hence are not included in each country’s table: GST (Goods and Services Tax), HST (Harmonized Sales Tax). These numbers are redacted as NUMERICAL_PII.

Asia Pacific

Australia

Identifier	PAI Label	Description	Example
Australian business number (ABN)	NUMERICAL_PII	A unique 11-digit identifier that every registered business in Australia is required to have	12345678901
Australian Company Number (ACN)	NUMERICAL_PII	A 9-digit number that must be displayed on all company documents	123 456 789
Bank-State-Branch (BSB)	ROUTING_NUMBER	A 6-digit number that identifies banks and branches across Australia	123-456
Tax File Number (TFN)	SSN	A 9-digit personal reference number used for tax and superannuation	456 789 123

China

Identifier	PAI Label	Description	Example
的医保卡号	HEALTHCARE_NUMBER	Healthcare number	Format varies by provider
纳税人识别号码	SSN	Taxpayer identification number consisting of 18 digits for individuals	463728374657483746

India

Identifier	PAI Label	Description	Example
Aadhaar	SSN	A 12-digit individual identification number used as a proof of identity and address	1234 5678 9123
Financial System Code	ROUTING_NUMBER	A unique 11-digit alphanumeric code that is used for online fund transfer transactions	IDIB000T131
Goods and Services Tax Identification Number (GSTIN)	NUMERICAL_PII	A unique 15-digit identification number assigned to every taxpayer in India	56HNJCA5424K1DM
Permanent Account Number (PAN)	SSN	A unique 10-digit tax identification number issued by the Income Tax Department	ABCJF54312D

Japan

Identifier	PAI Label	Description	Example
健康保険番号	HEALTHCARE_NUMBER	Health insurance number	Format varies by provider
マイナンバー (個人番号)	SSN	My Number (also known as "personal number"), a unique 12-digit number assigned to every resident of Japan, whether Japanese or foreign	123456789888

Korea

Identifier	PAI Label	Description	Example
건강보험증번호	HEALTHCARE_NUMBER	Health insurance card number	Format varies by provider
주민등록번호	SSN	Resident Registration Number used for tax purposes, consists of 13 digits	1236547898745

New Zealand

Identifier	PAI Label	Description	Example
Inland Revenue Department number (IRD)	SSN	A nine-digit individual identification number issued to each person by the New Zealand Inland Revenue Department, also known as a ‘tax file number’	099-999-999

Philippines

Identifier	PAI Label	Description	Example
PhilHealth ID number	HEALTHCARE_NUMBER	12-digit healthcare identification number	11-455678912-3
Social Security System number (SSS)	SSN	10-digit number used for tax purposes	12-3456789-1
Tax Identification Number (TIN)	SSN	12-digit number identifying a taxpayer	123 456 789 002

Europe

Identifier	PAI Label	Description	Example
Value-Added Tax (VAT)	NUMERICAL_PII	A tax applied to all goods and services that are bought and sold for use or consumption in the European Union, formatted as 2 letters (country code) followed by 8-10 digits. Localized names include French "numéro TVA"	DK99999999
International Bank Account Number (IBAN)	BANK_ACCOUNT	An international system of identifying bank accounts across national borders, consists of up to 34 alphanumeric characters including country codes	IE12BOFI90000112345678

Belgium

Identifier	PAI Label	Description	Example
Identificatienummer van de Sociale Zekerheid (INSZ) / Numéro d'identification à la sécurité sociale (NISS)	SSN	National identification number for social security, an 11-digit national registration number, the first 6 digits indicating date of birth	99013187654
Ondernemingsnummer	NUMERICAL_PII	A unique 10-digit identification number for a business	1987654323
Belasting Toegevoegde Waarde Nummer (BTW Nummer)	NUMERICAL_PII	An identification number for businesses used for VAT (Value Added Tax) purposes, formated as 2 letters followed by 10 digits	BE0784732737

Germany

Identifier	PAI Label	Description	Example
Bankkontonummer	BANK_ACCOUNT	Bank account number, 10 digits	0532013000
Krankenversicherungs nummer (KVNR)	HEALTHCARE_NUMBER	An alphanumeric code used for personal identification in Germany's national health insurance system (Krankenversicherung)	A123456789
Sozialversicherungs nummer	SSN, HEALTHCARE_NUMBER	A 12-digit number used to track a person's social security contributions, doubles as a healthcare number	12 123456 A 123
Steuer-Identnummer (St-Nr)	SSN	A unique 11-digit number assigned to every taxpayer in Germany by the Federal Central Tax Office	12345678909

France

Identifier	PAI Label	Description	Example
Numéro d'Inscription au Répertoire (NIR)	SSN	A 15-digit ID number commonly known as a numéro de sécurité sociale, also referred to as an Insee number, used for employment and French health benefits	1790223354367-97
Simplification des procedures d’Imposition (SPI)	SSN	French numéro de fiscal or a numéro SPI, a unique 13-digit tax number issued by the French tax authorities to all residents and non-residents with an obligation to pay tax	12 34 567 891 234
Système d'identification du répertoire des entreprises (SIREN)	NUM_PII	A 9-digit identifier assigned to every registered business in France by the National Institute of Statistics and Economic Studies (INSEE)	732 829 320

Italy

Identifier	PAI Label	Description	Example
Numero di identificazione fiscale or codice fiscale	SSN	Tax Identification Number (TIN), a 9-12 digit numeric code	000–123–456–789

Netherlands

Identifier	PAI Label	Description	Example
Burgerservicenummer (BSN)	SSN	A 9-digit citizen service number	123456789

Portugal

Identifier	PAI Label	Description	Example
Número de Identificação da Segurança Social (NISS)	SSN	An 11-digit number used to identify individuals in the Portuguese social security system	12354687985

Russia

Identifier	PAI Label	Description	Example
Идентификационный номер налогоплательщика (ИНН)	SSN	Taxpayer Personal Identification Number, 10-12 digits, used as a social security number	12 34567891 23

Spain

Identifier	PAI Label	Description	Example
Número de la Seguridad Social (NUSS)	SSN	11-12 digit social security number	12 34567891 23
Número de Identificación Fiscal (NIF)	SSN	A 10-character number that is used to interact with the Spanish tax agency	X12345678A

Ukraine

Identifier	PAI Label	Description	Example
Ідентифікаційний номер платника податків (ІНПП)	SSN	A 10-digit Taxpayer Identification Number (TIN)	1234567891

United Kingdom

Identifier	PAI Label	Description	Example
National Insurance Number (NIN)	SSN	Used in the UK's social security system and tax system, formatted as 2 prefix letters, 6 digits, and 1 suffix letter	QQ123456C
Sort code	ROUTING_NUMBER	Identifies both the bank (in the first digit or first two digits) and the branch where the account is held, usually formatted as 3 pairs of numbers	12-34-56
U.K. Unique Taxpayer Reference Number (UTR)	SSN	A 10-digit number, also called "tax reference," used in the U.K. when submitting a tax return	12345 67890

North and South America

Brazil

Identifier	PAI Label	Description	Example
Cadastro de Pessoas Físicas (CPF number)	SSN	Natural Persons Register, an 11-digit number in the format: 000.000.000-00	657.454.244-54

Canada

Identifier	PAI Label	Description	Example
Healthcare number	HEALTHCARE_NUMBER	Canadian Health Service Numbers, such as such as Care card number, OHIP, etc., required for access to healthcare benefits	Format varies by province
Numéro d'assurance sociale (NAS)	SSN	A 9-digit number that citizens and permanent residents need to work and be paid in Québec; French equivalent of SIN (see below)	365 789 654
Régie de l'assurance maladie du Québec (RAMQ)	HEALTHCARE_NUMBER	The Québec Health Insurance Number	BOUF 9401 1419
Social Insurance Number (SIN)	SSN	A 9-digit number that citizens and permanent residents need to work and be paid in Canada	321 654 987
Sort code	ROUTING_NUMBER	A unique 9-digit code that identifies the financial institution (4 digits) and branch of account (5-digit Transit Code)	123456789

Mexico

Identifier	PAI Label	Description	Example
Número de Seguridad Social (NSS)	SSN	Social Security Number, an 11-digit code	12345678912

United States

Identifier	PAI Label	Description	Example
Healthcare number	HEALTHCARE_NUMBER	A unique number assigned by a health insurance provider (includes private and government)	Format varies by provider
Social Security Number (SSN)	SSN	A 9-digit number issued to U.S. citizens, permanent residents, and (working) temporary residents	453-65-4543
U.S. Individual Taxpayer Identification Number (ITIN)	SSN	A 9-digit tax processing number that begins with "9" issued for some categories of population instead of SSN	923-45-6789

Docs: Redaction Languages

Mon, 01 Jan 0001 00:00:00 +0000

Cobalt features core support for 14 languages and extended support for 39 additional languages, with core languages featuring the highest level of performance. The complete list of supported languages below details which languages have core support, which have extended or beta support, and which are upcoming additions. New languages are continually being added, please contact us if you require a language not in the list below.

In addition to supporting 50+ languages, Cobalt offers support for regional language varieties in recognition of the large differences in vocabulary and grammar that can exist in the same language when spoken in different regions. So far, this includes support for varieties of English (US, UK, Canada and Australia), Spanish (Spain and Mexico), French (France and Canada), and Portuguese (Portugal and Brazil). Cobalt also supports code-switching, or mixing of different languages. This means that, in a phrase such as J’ai payé 76,88RM por ein Haarschnitt da 范玉菲 habang ko ay nasa Україна, multilingual PII is accurately de-identified. The selection of supported regional language varieties is continually being expanded, please let us know if there is a specific request.

Cobalt’s supported entity types function across each supported language, with multilingual equivalents of different PII (Personally Identifiable Information) entities, PHI (Protected Health Information) entities, and PCI (Payment Card Industry) entities being detected in each language. Our Supported Entity Types page provides a more detailed look at our coverage of language and region-specific entity equivalents. The solution is also sensitive to cross-linguistic differences in how names are structured, how place names are referred to, and how monetary units are described in different languages, among other differences.

Core Support

Language	ISO Code	Supported Regional Varieties	Support Level	Text Support	Audio Support	File Support	Labels
Dutch	nl	The Netherlands	Core	✓	✓	✓	✓
English	en	Australia, Canada, United Kingdom, United States	Core	✓	✓	✓	✓
French	fr	Canada (Quebec), France, Switzerland	Core	✓	✓	✓	✓
German	de	Germany, Belgium, Austria, Switzerland	Core	✓	✓	✓	✓
Hindi	hi	India	Core	✓	✓		✓
Italian	it	Italy, Switzerland	Core	✓	✓	✓	✓
Japanese	ja	Japan	Core	✓	✓	✓	✓
Korean	ko	Korea	Core	✓	✓		✓
Mandarin (simplified)	zh-Hans	China, Singapore	Core	✓	✓	✓	✓
Portuguese	pt	Brazil, Portugal	Core	✓	✓	✓	✓
Russian	ru	Russia	Core	✓	✓		✓
Spanish	es	Mexico, Spain	Core	✓	✓	✓	✓
Tagalog	tl	Philippines	Core	✓			✓
Ukrainian	uk	Ukraine	Core	✓	✓		✓

Extended Support

Language	ISO Code	Support Level	Text Support	Audio Support	File Support
Afrikaans	af	Extended	✓
Arabic	ar	Extended	✓
Bambara	bm	Extended	✓
Bengali	bn	Extended	✓
Belarusian	be	Extended	✓
Bulgarian	bg	Extended	✓
Burmese	my	Extended	✓
Cantonese (traditional)	zh-Hant	Extended	✓
Catalan	ca	Extended	✓
Croatian	hr	Extended	✓
Czech	cs	Extended	✓
Danish	da	Extended	✓
Estonian	et	Extended	✓
Finnish	fi	Extended	✓
Georgian	ka	Extended	✓
Greek	el	Extended	✓
Hebrew	he	Extended	✓
Hungarian	hu	Extended	✓
Icelandic	is	Extended	✓
Indonesian	id	Extended	✓	✓
Khmer	km	Extended	✓
Latvian	lv	Extended	✓
Lithuanian	lt	Extended	✓
Luxembourgish	lb	Extended	✓
Malay	ms	Extended	✓
Moldovan	ro	Extended	✓
Norwegian (Bokmål)	nb	Extended	✓	✓
Persian (Farsi)	fa	Extended	✓
Polish	pl	Extended	✓	✓	✓
Punjabi	pa	Extended	✓
Romanian	ro	Extended	✓
Slovak	sk	Extended	✓
Slovenian	sl	Extended	✓
Swahili	sw	Extended	✓
Swedish	sv	Extended	✓	✓
Tamil	ta	Extended	✓
Thai	th	Extended	✓
Turkish	tr	Extended	✓	✓
Vietnamese	vi	Extended	✓

Docs: Prerequisites and System Requirements

Mon, 01 Jan 0001 00:00:00 +0000

Prerequisites

Info

Please only run one container instance per machine. Running multiple containers results in vastly reduced performance.

The following prerequisites are required to run the container:

Container engine, such as Docker (can be installed using the official instructions )
(GPU only) Nvidia Container Toolkit with Nvidia driver version 515 or higher (can be installed using the following installation guide )

All other dependencies, such as CUDA are included with the container and don’t need to be installed separately.

System Requirements

Docker and docker-compose installed.

The image comes in two different build flavours:

A compact, CPU-only container that runs on any Intel or AMD CPU and a container with GPU acceleration. The CPU container is highly optimised for the majority of use cases, as the container uses hand-coded AMX/AVX2/AVX512/AVX512 VNNI instructions in conjunction with Neural Network compression techniques to deliver a ~25X speedup over a reference transformer-based system.
A GPU container is designed for large-scale deployments making billions of API calls or processing terabytes of data per month.

Minumum Requirements

The minimum system requirements for the container image are as follows:

	Minimum	Recommended (Text only)	Recommended (All Features)	Recommended Concurrency
CPU	Any x86 (Intel or AMD) processor with 7.5GB free RAM and 50GB disk volume	Intel Sapphire Rapids or newer CPUs supporting AMX with 16GB RAM and 50GB disk volume	Intel Sapphire Rapids or newer CPUs supporting AMX with 64GB RAM and 100GB disk volume	1
GPU	Any x86 (Intel or AMD) processor with 28GB free RAM. Nvidia GPU with compute capability 7.0 or higher (Volta or newer) and at least 16GB VRAM. 100GB disk volume	Any x86 (Intel or AMD) processor with 32GB RAM and Nvidia Tesla T4 GPU. 100GB disk volume	Any x86 (Intel or AMD) processor with 64GB RAM and Nvidia Tesla T4 GPU. 100GB disk volume	32

Recommended Requirements

While CPU-based container will run on any x86-compatible instance, the below cloud instance types give optimal throughput and latency per dollar:

Platform	Recommended Instance Type (Text only)	Recommended Instance Type (All Features)
Azure	Standard_E2_v5 (2 vCPUs, 16GB RAM)	Standard_E8_v5 (8 vCPUs, 64GB RAM)
AWS	M7i.large (2 vCPUs, 8GB RAM)	m7i.4xlarge (16 vCPUs, 64GB RAM)
GCP	N2-Standard-2 (2 vCPUs, 8GB RAM)	N2-Standard-16 (16 vCPUs, 64GB RAM)

Notes:

In the event when a lower latency is required, the instance type should be scaled; e.g. using an M7i.xlarge in place of a M7i.large. While the Cobalt docker solution can make use of all available CPU cores, it delivers best throughput per dollar using a single CPU core machine. Scaling CPU cores does not result in a linear increase in performance.

Similarly for the GPU-based image, it is recommended the following Nvidia T4 GPU-equipped instance types:

Platform	Recommended Instance Type (Text only)	Recommended Instance Type (All Features)
Azure	Standard_NC8as_T4_v3	Standard_NC8as_T4_v3
AWS	G4dn.2xlarge	G4dn.4xlarge
GCP	N1-Standard-8 + Tesla T4	N1-Standard-16 + Tesla T4

Docs: Proto API Reference

Mon, 01 Jan 0001 00:00:00 +0000

The API is defined as a protobuf spec, so native bindings can be generated in any language with gRPC support. We recommend using buf to generate the bindings.

This section of the documentation is auto-generated from the protobuf spec. The service contains the methods that can be called, and the “messages” are the data structures (objects, classes or structs in the generated code, depending on the language) passed to and from the methods.

TrifidService

Service that implements the Cobalt Trifid Redaction Engine API.

Version

Version(VersionRequest) VersionResponse

Returns version information from the server.

ListModels

ListModels(ListModelsRequest) ListModelsResponse

ListModels returns information about the models the server can access.

RedactText

RedactText(RedactTextRequest) RedactTextResponse Redact text using a redaction engine that is configured with the provided redaction configuration.

RedactTranscript

RedactTranscript(RedactTranscriptRequest) RedactTranscriptResponse

redacts transcript using a redaction engine that is configured with the provided redaction configuration.

StreamingRedactTranscribedAudio

StreamingRedactTranscribedAudio(StreamingRedactTranscribedAudioRequest) StreamingRedactTranscribedAudioResponse

Performs bidirectional streaming redaction on transcribed audio. Receive redacted audio while sending audio. The transcription of audio data must be ready before sending the audio.

StreamingTranscribeAndRedact

StreamingTranscribeAndRedact(StreamingTranscribeAndRedactRequest) StreamingTranscribeAndRedactResponse

Performs bidirectional streaming speech recognition and redaction. Receive redacted audio and transcriptions while sending audio.

Messages

If two or more fields in a message are labeled oneof, then each method call using that message must have exactly one of the fields populated
If a field is labeled repeated, then the generated code will accept an array (or struct, or list depending on the language).

ListModelsRequest

The top-level message sent by the client for the ListModels method.

ListModelsResponse

The message returned to the client by the ListModels method.

Fields

models (ModelInfo repeated) List of models available for use on Trifid server.

ModelInfo

Description of a Trifid Model

Fields

id (string ) Unique identifier of the model. This identifier is used to choose the model that should be used for recognition, and is specified in the RedactionConfig message.
name (string ) Model name. This is a concise name describing the model, and may be presented to the end-user, for example, to help choose which model to use for their recognition task.
redaction_classes (string repeated) List of supported redaction classes.

RedactTextRequest

The top-level message sent by the client for the ListModels method.

Fields

redaction_config (RedactionConfig)
text (string ) Unique identifier of the model. This identifier is used to choose the model that should be used for recognition, and is specified in the RedactionConfig message.

RedactionConfig

Configuration for setting up a redaction engine.

Fields

model_id (string) Unique identifier of the model to use, as obtained from a ModelInfo message.
redaction_classes (string repeated) List of whitelisted redaction classes. If the list is empty, server default redaction class list will be considered.
disable_streaming (bool ) This is an optional field. If this is set to true, Cobalt Privacy Screen will redact entire transcript at once, by doing so, redaction accuracy will increase at the cost of higher latency. If set to false, Cobalt Privacy Screen will redact one utterance at a time and return the result as soon as possible. The default is false.
custom_classes (CustomClasses repeated) This is an optional field. If set, then provided list will be used to extend the list of redaction classes.

CustomClasses

CustomClass allows the client to define a new redaction class. Patterns defined here will override default redaction class for matching tokens.

Fields

redaction_class string This is name of the new redaction class. For example, this could be “COMPANY_NAME”.
pattern string Pattern defines a Python regex expression that would be used to identify tokens in text that get redacted to this new redaction class. For example, “COBALT|GOOGLE|MICROSOFT”, or more complex patterns such as “^COMPANY-[\d]{4}$”.

ListModelsResponse

The message returned to the client by the ListModels method.

Fields

models (ModelInfo repeated) List of models available for use on Trifid server.

ModelInfo

Description of a Trifid Model

Fields

id (string ) Unique identifier of the model. This identifier is used to choose the model that should be used for recognition, and is specified in the RedactionConfig message.
name (string ) Model name. This is a concise name describing the model, and may be presented to the end-user, for example, to help choose which model to use for their recognition task.
redaction_classes (string repeated) List of supported redaction classes.

RedactTranscriptRequest

The top-level messages sent by the client for the RedactTranscript method. Contains redaction config and a transcription to redact.

Fields

config (RedactionConfig ) redaction config
transcript (Transcript ) transcription to redact

RedactTranscriptResponse

The top-level message sent by the server for the RedactTranscript method. Contains redacted transcript.

Fields

transcript (Transcript )

RedactionConfig

Configuration for setting up a redaction engine.

Fields

model_id (string ) Unique identifier of the model to use, as obtained from a ModelInfo message.
redaction_classes (string repeated) List of whitelisted redaction classes. If the list is empty, server default redaction class list will be considered.
disable_streaming (bool ) This is an optional field. If this is set to true, Trifid will redact entire transcript at once, by doing so, redaction accuracy will increase at the cost of higher latency. If set to false, Trifid will redact one utterance at a time and return the result as soon as possible. The default is false.

Transcript

Transcript contains multiple utterance of the audio.

Fields

utterances (Utterance repeated)

Utterance

Utterance of the audio

Fields

text (string ) Text representing the utterance of the audio.
audio_channel (uint32 ) Channel of the audio file associate with this utterance. Channels are 0-indexed, so the for mono audio data, this value will always be 0.
start_time_ms (uint64 ) Time offset in milliseconds relative to the beginning of audio corresponding to the start of this utterance.
duration_ms (uint64 ) Duration in milliseconds of the current utterance in the audio.
asr_confidence (double ) ASR confidence estimate between 0 and 1. A higher number represents a higher likelihood of the output being correct.
words_info (WordInfo repeated) Word-level information corresponding to the utterance. This field contains word-level timestamps, which are essential as input for audio redaction. This field is only available in an output utterance if enable_word_info was set to true in the RedactionConfig.

StreamingRedactTranscribedAudioRequest

The top-level messages sent by the client for the StreamingRedactTranscribedAudio method. In this streaming call, multiple StreamingRedactTranscribedAudioRequest messages should be sent. The first message must contain a RedactTranscribedAudioConfig message only and all subsequent messages must contain audio data only.

Fields

oneof request.config (RedactTranscribedAudioConfig)
oneof request.audio (bytes)

RedactTranscribedAudioConfig

Configuration for setting up a StreamingRedactTranscribedAudio method.

Fields

redaction_config(RedactionConfig) Text redaction config.
transcript(Trancript) Transcription of the entire audio. This must be ready before sending the audio.

StreamingRedactTranscribedAudioResponse

The top-level message sent by the server for the StreamingRedactTranscribedAudio method. In this streaming call, multiple StreamingRedactTranscribedAudioResponse messages contain either Utterance or redacted audio data will be returned.

Fields

oneof result.utterance (Utterance)
oneof result.audio (bytes)

StreamingTranscribeAndRedactRequest

The top-level messages sent by the client for the StreamingTranscribeAndRedact method. In this streaming call, multiple StreamingTranscribeAndRedactRequest messages should be sent. The first message must contain a TranscribeAndRedactConfig message only and all subsequent messages must contain audio data only.

Fields

oneof request.config (TranscribeAndRedactConfig)
oneof request.audio (bytes)

TranscribeAndRedactConfig

Configuration for setting up a StreamingTranscribeAndRedact method.

Fields

redaction_config(RedactionConfig) Text redaction config.
enable_unredacted_transcript(bool) This is an optional field. If this is set to true, each utterance result will include unredacted utterance. If set to false, no unredacted utterance will be returned. The default is false.

StreamingTranscribeAndRedactResponse

The top-level message sent by the server for the StreamingTranscribeAndRedact method. In this streaming call, multiple StreamingTranscribeAndRedactResponse messages contain either TranscribeAndRedactUtterance or redacted audio data will be returned.

Fields

oneof result.utterance (Utterance)
oneof result.audio (bytes)

VersionRequest

The top-level message sent by the client for the Version method.

VersionResponse

The top-level message sent by the server for the Version method.

Fields

version (string ) Version of the server handling these requests.

WordInfo

Word level details for words in a utterance.

Fields

text (string ) The actual word corresponding to the utterance.
asr_confidence (double ) ASR confidence estimate between 0 and 1. A higher number represents a higher likelihood that the word was correctly recognized.
start_time_ms (uint64 ) Time offset in milliseconds relative to the beginning of audio received by the recognizer and corresponding to the start of this spoken word.
duration_ms (uint64 ) Duration in milliseconds of the current word in the spoken audio.
is_redacted (bool ) If this is set to true, it denotes that the curent word is redacted word or an original word of a redacted word.
redaction_class (string ) Recognized redaction class. This is available only if the current word is a redacted word.
redaction_confidence (double ) Redactio confidence estimate between 0 and 1. A higher number represents a higher likelihood that the word was correctly recognized. This is available only if the current word is a redacted word.

Enums

Scalar Value Types

.proto Type	C++ Type	C# Type	Go Type	Java Type	PHP Type	Python Type	Ruby Type
double	double	double	float64	double	float	float	Float
float	float	float	float32	float	float	float	Float
int32	int32	int	int32	int	integer	int	Bignum or Fixnum (as required)
int64	int64	long	int64	long	integer/string	int/long	Bignum
uint32	uint32	uint	uint32	int	integer	int/long	Bignum or Fixnum (as required)
uint64	uint64	ulong	uint64	long	integer/string	int/long	Bignum or Fixnum (as required)
sint32	int32	int	int32	int	integer	int	Bignum or Fixnum (as required)
sint64	int64	long	int64	long	integer/string	int/long	Bignum
fixed32	uint32	uint	uint32	int	integer	int	Bignum or Fixnum (as required)
fixed64	uint64	ulong	uint64	long	integer/string	int/long	Bignum
sfixed32	int32	int	int32	int	integer	int	Bignum or Fixnum (as required)
sfixed64	int64	long	int64	long	integer/string	int/long	Bignum
bool	bool	bool	bool	boolean	boolean	boolean	TrueClass/FalseClass
string	string	string	string	String	string	str/unicode	String (UTF-8)
bytes	string	ByteString	[]byte	ByteString	string	str	String (ASCII-8BIT)