Mining Asymmetric Intertextuality

• •

  Literature: Many pre-modern novels or poems reference or allude to classical works (e.g., Shakespeare, Homer, or Chinese classics like the Shijing). However, these allusions are one-sided: the original works remain static and unchanged, while the newer texts reinterpret or recontextualize them [4].

• •

  Journalism: News articles frequently quote or paraphrase statements from public figures or historical documents. However, these quotes are often taken out of context or reframed to fit the narrative of the article, making the relationship non-reciprocal and often asymmetric.

Detecting one-way intertextuality is challenging because the references are often implicit. Traditional token-based methods (like keyword matching or n-gram analysis) struggle to capture paraphrased or recontextualized references, while semantic search methods often miss subtle thematic or structural borrowings [2].

For example, a modern text may paraphrase an older work without using direct quotations, making it hard for basic search algorithms to detect the connection. Similarly, thematic borrowings, where ideas or concepts are reinterpreted without explicit citation, pose a major challenge for current intertextuality detection tools.

Not all one-way relationships are asymmetric, but many are. In cases where a newer text transforms, paraphrases, or recontextualizes an older one, the relationship between the texts becomes structurally non-isomorphic. The newer text may borrow heavily from the older one, but the transformation of the content introduces asymmetry. The two texts are no longer equivalent in their relationship, and the influence is not easily mapped in a bi-directional way.

This is particularly true in cases of paraphrasing or thematic borrowing, where the newer text changes the meaning or application of the original content. The lack of direct quotation or citation makes these connections harder to detect, requiring more advanced systems that can capture the semantic and structural differences between the two texts.

Asymmetric intertextuality can manifest in various forms, each requiring different detection strategies. These intertextual relationships can be classified into three broad categories—Quotation, Citation, and Implicit Intertextuality—each reflecting a different type of textual borrowing or influence.

This classification highlights the different ways in which intertextuality can manifest. Identifying and processing these relationships requires the ability to break down sentences into their core elements, as discussed further in the next section.

In detecting asymmetric intertextuality, the system uses various external cues—such as quotation marks, citation markers, and paraphrasing cues—that signal different types of intertextual references. These external elements are unequipped during preprocessing but are stored as metadata. They are crucial for both metadata-filtered vector search and deep search, helping to refine candidate matches and ultimately re-equipping elements when necessary.

Below, we outline the specific unequippable elements and associated metadata for each intertextuality type. Both the vector search and deep search stages utilize these elements to identify asymmetric relationships such as matching similar author names or paraphrased ideas across documents.

The system unequips these external cues during preprocessing and stores them in metadata. These elements are used during both metadata-filtered vector search and deep search to guide the search, refine candidate matches, and ultimately reintroduce these elements when necessary.

In addition to the intertextuality-type-specific elements, there are universal elements that apply across different types of intertextuality. These elements are potentially useful for defining more nuanced asymmetric intertextuality patterns in future research and are particularly useful during deep search for refining matches and understanding the context. While universal re-equippable elements may not always directly indicate a specific intertextuality type, they help to enhance the search and provide additional context for implicit and asymmetric intertextuality patterns.

Below is a comprehensive list of universal re-equippable elements, along with their associated metadata, which is used to reintroduce them during the deep search phase.

By storing these unequipped elements as metadata, we ensure that the original structure is accurately restored during deep search, allowing for a more nuanced understanding of intertextual relationships. These elements play a crucial role in refining the results and improving the detection of both explicit and implicit intertextuality.

At the document level, the system must account for the multi-layered structure of texts by identifying and tagging discourse roles (e.g., introduction, methods, results). This process helps capture the broader context in which citations, quotations, and paraphrases occur [29, 30].

The following strategies could be employed:

• •

  Discourse Tagging: Each section of the document is assigned a specific discourse role (e.g., background, methodology, or conclusion). This helps in contextualizing the nature of intertextual relationships, such as whether a citation in the methodology section indicates a methodological borrowing or whether a quotation in the discussion section suggests a result comparison [29, 30].

• •

  Hierarchical Decomposition: Documents are broken down into sections, paragraphs, and sentences, while preserving their hierarchical structure. This decomposition is crucial for detecting cross-document influences, where intertextual relationships may span entire sections or even whole documents, rather than being restricted to sentence-level interactions [5].

At the sentence level, the system transforms each sentence into a structured, vectorized representation that captures its semantic meaning independently of surface-level markers like quotation marks or citation tags. This is particularly important for detecting implicit intertextuality, where paraphrasing or indirect references obscure the direct connection between texts [31].

• •

  Sentence Embeddings: Sentences are encoded into vector embeddings (e.g., using Sentence-BERT [32]) to capture their semantic content. These embeddings support semantic similarity searches in the merge/hybrid search phase, allowing the system to detect paraphrased or reworded intertextual connections that lack lexical overlap with the source text [33].

• •

  Contextual Metadata: Each sentence is enriched with metadata such as discourse role, citation context, and document-level identifiers. This metadata is essential for the next step, enabling the system to filter and refine the search based on specific types of intertextual relationships (e.g., direct quotations, paraphrasing, or implicit citations) [5].

While sentence-level analysis provides fine-grained detection of intertextuality, it is often insufficient when dealing with implicit citations or thematic borrowing across larger sections or whole documents. To address this, the system must establish cross-document links at both the sentence and section levels [31].

• •

  Linking Documents for Implicit Citations: Many intertextual connections, especially in academic writing, involve implicit references where one document builds upon another without explicitly citing it. By linking documents based on semantic similarity and discourse roles, the system can detect such implicit relationships [5, 31].

• •

  Cross-Sentence Dependencies: Some intertextual relationships, such as implicit quotations, may require analyzing surrounding sentences or even entire sections. By preserving the hierarchical structure of documents, the system allows for cross-sentence and cross-section comparisons during the merge phase, which is critical for detecting complex intertextual influences [29].

To support the hybrid search in the next step, the normalized data must be stored in a database that allows for efficient querying and retrieval. This involves two key steps:

• •

  Vectorization: Each sentence and section is vectorized (such as Cohere-embed-english-v3.0) and stored in a vector database (e.g., FAISS or Milvus), which allows for fast nearest-neighbor searches during the merge phase [34]. These vectors enable the system to perform semantic comparisons across documents with minimal computational overhead, even in large corpora [33].

• •

  Metadata Storage: In addition to vectorized representations, each document and sentence is stored alongside its corresponding metadata, including discourse role, citation type, and quotation markers [29, 5]. This metadata is crucial for hybrid search filtering, where the system can refine its results based on the type of intertextual relationship being queried (e.g., direct quotation, method citation, paraphrase).

By organizing the normalized data in this way, the system ensures that the subsequent merge phase can leverage both semantic similarity and metadata filtering to detect asymmetric intertextual relationships across documents [27].

Each sentence in the Sentence Collection is linked to its parent document using the document_id field. Additionally, each sentence is linked to a specific section within the document using the section_id field, ensuring that the context of the sentence is preserved.

Once a potential intertextual match is detected, the system re-equips the external cues that were stripped during preprocessing. These unequippable elements, stored as metadata, are critical for refining the detected relationship and ensuring that the intertextual connection is valid. The re-equipped elements are reintroduced in two stages:

• •

  Stage 1 - Metadata-filtered Vector Search: Elements stored as metadata are reintroduced based on the type of intertextuality detected (e.g., direct quote, paraphrase, citation). This ensures that the original structure of the text is restored and that the relationship is presented in its full context.

• •

  Stage 2 - Deep Search: The system uses a large language model (LLM) to process the re-equipped elements in context, ensuring that the final output is coherent and semantically aligned with the original text. The LLM also verifies the relationship by cross-referencing the re-equipped elements with both the source and target documents.

The specific elements that are re-equipped include:

• •

  Quotation Marks $(\mathbf{1}_{Q_{i}})$: Reintroduced for sentences identified as direct quotes to distinguish verbatim reuse of text. These are essential for identifying Direct Quotes (DQ) and Reported Quotes (RQ).

• •

  Citation Markers $(\mathbf{1}_{C_{i}})$: Restored for both explicit and implicit citations, marking the source of the referenced material. Citation markers are critical for identifying Method Citations (MC), Result Comparisons (RC), and Background Citations (BC).

• •

  Paraphrasing Cues $(\mathbf{1}_{P_{i}})$: Phrases like "according to" or "as mentioned by" are restored to clarify paraphrased content. These are especially important for detecting Indirect Quotes (IQ) and Implicit Citations (ICT).

• •

  Speaker Attribution $(\mathbf{1}_{A_{i}})$: Re-applied for reported quotes, indicating the original speaker or source of the information. This is important for Reported Quotes (RQ) and cases where the original speaker is referenced but not directly quoted.

• •

  Discourse Cues $(\mathbf{1}_{D_{i}})$: Transitional phrases and sentence starters (e.g., "however", "therefore", "according to") are restored to provide the rhetorical structure of the text, which is useful for Indirect Quotes (IQ) and Implicit Citations (ICT).

• •

  Figures and Tables $(\mathbf{1}_{F_{i}})$: Mentions of figures or tables (e.g., "see Fig. 3") and their captions are restored when they play a role in the intertextual relationship, particularly in Result Comparisons (RC).

• •

  Numerical Data $(\mathbf{1}_{N_{i}})$: Relevant dates, numbers, and statistical results are reintroduced to provide factual context, especially in citations involving data or analysis, such as Method Citations (MC) or Result Comparisons (RC).

These re-equipped elements are represented by indicator functions in the mathematical formalism, which adjust the similarity score based on whether these external cues were present in the original sentence. The indicator functions $\mathbf{1}_{X_{i}}$ represent the presence of each element, where $X$ can be any unequippable element (e.g., $Q_{i}$ for quotation marks, $C_{i}$ for citation markers, etc.).

The re-equipping process ensures that the semantic meaning of the text is maintained while providing the necessary context to validate the intertextual relationship. The following mathematical formalism describes how these elements are used to modify the similarity score:

In this section, we formalize the process of re-equipping external elements such as quotation marks, citation markers, and paraphrasing cues when detecting intertextual relationships. The general equation below adjusts the similarity score between a source chunk and a target document by accounting for the presence of these external elements.

In the candidate subsetting stage, we compute the cosine similarity between vectorized chunks or documents. We define two equations for the vector search process:

• •

  Quotation Types (Direct Quote, Indirect Quote, Reported Quote): This involves comparing two chunks of text, typically at the sentence-to-sentence level:

  $$\text{Similarity}_{\text{Quotation}}(c_{i},c_{j})=\cos(c_{i},c_{j})$$

  Where $c_{i}$ and $c_{j}$ are chunks of text, and $\cos(c_{i},c_{j})$ represents the cosine similarity between their vector embeddings.

• •

  Citation Types (Method Citation, Result Comparison, Background Citation): This involves comparing a chunk to a document, typically at the sentence-to-document level:

  $$\text{Similarity}_{\text{Citation}}(c_{i},D_{j})=\cos(c_{i},D_{j})$$

  Where $c_{i}$ is a chunk of text and $D_{j}$ is a document or a section of the document. The equation computes the cosine similarity between the chunk and the document.

The adjusted similarity between a chunk $c_{i}$ from the source document and a document $D_{j}$ from the target corpus is given by:

Where:

• •

  $\cos(\mathbf{v}(c_{i}),\mathbf{v}(D_{j}))$ is the base cosine similarity between the vector embeddings of chunk $c_{i}$ and document $D_{j}$.

• •

  $\lambda_{X}$ is the weight associated with the external element $X$ (e.g., quotation marks, citation markers).

• •

  $\mathbf{1}_{X_{i}}$ is an indicator function that equals 1 if the external element $X$ is present in the chunk $c_{i}$, and 0 otherwise.

This formalism helps prioritize different types of intertextuality by adjusting the similarity score based on the presence of specific re-equippable elements.

The deep search phase refines the results by reintroducing the universal re-equippable elements and applying LLM (Large Language Model) processing to enhance the search. We split the deep search into two stages:

• •

  Using Vector Databases: The system employs specialized vector databases, such as FAISS or Milvus [34, 35], designed to handle high-dimensional similarity searches efficiently. These databases are optimized for fast nearest-neighbor searches, allowing quick retrieval of semantically similar text chunks from large corpora.

• •

  High-Dimensional Space Optimization: The system should be able to performs efficient vector comparisons in high-dimensional space, facilitating rapid searches across vast document collections without performance degradation.

• •

  Real-Time Retrieval: As new documents are added to the corpus, the system utilizes the vector database to perform real-time retrieval of semantically similar text. This ensures that the system can scale effectively while maintaining low query latency [34].

• •

  Incremental Vectorization: New documents are vectorized incrementally as they are introduced, eliminating the need to reprocess the entire corpus, which enhances the system’s responsiveness and scalability [27].

• •

  Seamless Integration of New Documents: The system dynamically integrates new documents into the search space, allowing continuous indexing and querying without disrupting the existing structure.

• •

  Avoiding Full Reprocessing: By vectorizing new documents dynamically, the system avoids the significant computational overhead typically associated with full corpus reprocessing. This is especially important for handling large, continuously growing datasets [33].

• •

  Sparse Attention Models: To handle large datasets efficiently, sparse attention models [36] are used to reduce the computational complexity of similarity searches, ensuring high accuracy while minimizing resource usage.

• •

  Divide-and-Conquer Approaches: The system employs divide-and-conquer strategies, breaking large corpora into smaller chunks for independent processing. This modular approach helps the system manage large datasets more efficiently without sacrificing accuracy.

• •

  Optimized Vector Databases: The use of optimized vector databases ensures that even large corpora are processed with minimal computational overhead. Extensive testing has demonstrated that the system maintains high accuracy and low latency as corpus size increases [34, 33].

In summary, by leveraging vector databases, dynamic updates, and efficient handling of large datasets, the system is designed to scale while maintaining its accuracy and performance. The next section will focus on a specific design implementation chosen for its balance of scalability, accuracy, and computational efficiency.

In this step, we split the legal documents into smaller chunks, such as sentences or sections. This process allows the system to analyze content at a finer granularity, which is essential for detecting paraphrasing, thematic borrowing, or other intertextual relationships, especially in legal texts.

For this demonstration, we use the Contract Understanding Atticus Dataset (CUAD), a specialized dataset designed to facilitate legal contract review. CUAD consists of 510 contracts and over 13,000 annotated clauses, specifically selected to highlight key clauses that are important for legal professionals.

Let’s consider a document with the following chunk:

  "EXHIBIT 4.25 INFORMATION IN THIS EXHIBIT IDENTIFIED BY [ * * * ]"

Using a sentence or section tokenizer, the document would be split into smaller chunks, preserving meaningful units of analysis. For example:

• •

  Chunk 1: "EXHIBIT 4.25 INFORMATION IN THIS EXHIBIT IDENTIFIED BY [ * * * ]"

• •

  Chunk 2: "This Agreement shall commence on the Effective Date and remain in effect until terminated as provided herein."

Each chunk is stored as a separate record in the database. Here’s an example of how the first chunk is stored:

{
  "_id": {
    "$oid": "670f89c9c6abf22788e288f5"
  },
  "doc_id": "670f881e870d0cb83f7e67ed",
  "filename": "ABILITYINC_06_15_2020-EX-4.25-SERVICES AGREEMENT.txt",
  "chunk_number": 1,
  "chunk_text": "EXHIBIT 4.25 INFORMATION IN THIS EXHIBIT IDENTIFIED BY [ * * * ]"
}

This splitting process enables finer-grained analysis, crucial for detecting intertextual relationships in legal contracts.

Next, we implement two runs of mining with a Large Language Model (LLM), utilizing few-shot prompting to guide the model in identifying relevant relationships and metadata:

• •

  Run 1: Intertextuality-related items. This run focuses on identifying intertextual relationships, such as direct quotations, paraphrasing, or thematic borrowing. The model uses a few-shot prompt to identify paraphrasing or thematic borrowing between chunks. See Appendix A.1 for the detailed prompt.

• •

  Run 2: Universal-related items. This run focuses on unequippable elements that apply across documents, such as citation markers, quotation marks, or discourse cues. The model uses a few-shot prompt to extract universal metadata, such as discourse roles and references. See Appendix A.2 for the detailed prompt.

By running these two passes with LLM-based few-shot prompting, we capture both specific intertextuality-related relationships and broader document structure elements, enabling a comprehensive analysis of legal documents.

After the splitting and enrichment of legal contract data (as discussed in Step 1), the next critical step is migrating the data from MongoDB to Elasticsearch and preparing it for hybrid search. This process involves deciding which fields require vector embeddings and which can be directly indexed for metadata-based filtering. The primary criterion for this decision is whether capturing the semantic meaning of the text is essential for improving search accuracy.

The Merge phase finalizes the intertextuality detection process by leveraging the Elasticsearch index created during the Normalize step. Here, we perform a metadata-filtered search to reintroduce unequipped elements and optionally carry out a deep search with LLM verification.