tokenization
Here are 32 public repositories matching this topic...
This is a java version of Chinese tokenization descried in BERT.
-
Updated
Nov 10, 2022 - Java
Identify and tokenize sensitive data automatically using Cloud DLP and Dataflow
-
Updated
Mar 24, 2025 - Java
Public code samples and resources for the Thales CipherTrust Application Protection products of the CipherTrust Data Security Platform
-
Updated
Apr 21, 2025 - Java
serverless ☁️ 🚀 , pseudonymizing proxy between Worklytics and your workplace 💼 SaaS data sources' APIs. Data Loss Prevention (DLP) 🛡️🔒 and compliance layer deployable to AWS Lambda or GCP Cloud Functions.
-
Updated
Apr 21, 2025 - Java
Babel Street Analytics Client Library for Java
-
Updated
Apr 15, 2025 - Java
🔧 My studies on context-free grammar, using ANTLR4 (C++) to generate the parser files. Some basics are developed, such as token processing, recursion, variable definition, array processing, Abstract Syntax Tree (AST) manipulation, UNICODE support, and error handling.
-
Updated
Oct 17, 2022 - Java
The tokenisation of spoken text. Received by the Watson STT and sent to the Apache OpenNLP. Additional code creates individual tokens, depending on the recorded sentences
-
Updated
Jul 16, 2018 - Java
Multilingual Natural Language Processing for Java
-
Updated
Dec 13, 2023 - Java
Resolving conflict merges with ASTs
-
Updated
Mar 18, 2017 - Java
IXOPAY SDK for card tokenization on Android.
-
Updated
Jan 31, 2025 - Java
Android Java "add payment card" form - This app demonstrates how simple it is to add payment card data to your app with VeryGoodSecurity
-
Updated
Oct 9, 2019 - Java
Java Springboot backend application for onboarding new customers (REST Apis) - Monolithic Application
-
Updated
Dec 18, 2024 - Java
Language processing interface: some tools to process different natural languages
-
Updated
Jul 28, 2017 - Java
Compiler Construction contains three phase 1)Lexical Analyzer 2)Syntax Analyzer 3)Semantic Analyzer
-
Updated
Apr 7, 2019 - Java
Program that preprocesses a collection of documents to calculate the frequency of the most common terms and identify the keywords of each document. The first time will do it without using the stemming technique and without removing the stopwords. The second time will use these techniques.
-
Updated
Sep 6, 2019 - Java
A take on building an interpreter (possibly a compiler) for a subset of Pascal
-
Updated
Feb 2, 2024 - Java
True Lexar, lexical analyser in java.....gui in javafx.
-
Updated
Mar 8, 2018 - Java
tokenization project
-
Updated
Jan 13, 2020 - Java
Tokenizer for Teragrep
-
Updated
Oct 2, 2024 - Java
Improve this page
Add a description, image, and links to the tokenization topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the tokenization topic, visit your repo's landing page and select "manage topics."