BibexPy

Harmonizing the Bibliometric Symphony of Scopus and Web of Science

Stable Features
  • Enhanced Data Processing
  • Intelligent Merging System
  • Automated Directory Management
  • Advanced Error Handling
  • Improved Data Validation
  • WoS & Scopus Support
Quick Start
git clone https://github.com/bcankara/BibexPy.git View Stable Version
Latest Features
  • Enhanced Data Processing
  • Intelligent Merging System
  • Automated Directory Management
  • Advanced Error Handling
  • Improved Data Validation
  • Multi-Database Support (7+ Sources)
  • PubMed, Dimensions, Cochrane
  • Lens, OpenAlex + WoS & Scopus
Quick Start
git clone -b Dev https://github.com/bcankara/BibexPy.git View Developer Version
Get Started
Data Collection
WoS & Scopus
Merge & Clean
Intelligent Processing
API Enrich
Metadata Enhancement
Analyze
Other Bibliometrics Software

Features

DOI-Based Deduplication

Intelligent Scientific Database Merge Component that combines and deduplicates data from Web of Science and Scopus using advanced bibliometric intelligence.

API-Driven Metadata

Automatically searches and enriches empty fields in merged data through 5+ API services, enhancing bibliographic databases with comprehensive metadata.

Format Conversion

Processes raw data files (WoS .txt and Scopus .csv) into standardized formats, creating enriched bibliographic outputs ready for analysis.

Prerequisites

Required Python Version

  • Python ≥ 3.9.0

Libraries

  • python-dotenv==1.0.0
  • pandas>=2.0.0
  • openpyxl>=3.1.2
  • numpy>=1.24.0
  • requests>=2.31.0
  • scikit-learn>=1.3.0
  • scipy>=1.11.0
  • tqdm>=4.65.0
  • xlrd>=2.0.1
  • xlsxwriter>=3.1.0
  • colorama>=0.4.6
  • typing-extensions>=4.7.0

Installation

  1. Clone the Repository
    git clone https://github.com/bcankara/BibexPy.git
  2. Navigate to the Directory
    cd BibexPy
  3. Install Dependencies
    pip install -r requirements.txt
  4. (Optional) Virtual Environment Setup
    python -m venv venv

# Mac/Linux
source venv/bin/activate

# Windows
venv\Scripts\activate

Usage

Project Structure
BibexPy/
├── .env                      # Environment variables for API keys
├── requirements.txt          # Python package dependencies
├── DataProcessor.py          # Main application file
├── Main/
│   ├── modules/
│   │   ├── __init__.py
│   │   ├── api_utils.py     # API integration utilities
│   │   ├── file_utils.py    # File handling utilities
│   │   ├── merge_utils.py   # Data merging operations
│   │   ├── post_process.py  # Post-processing utilities
│   │   └── stats_utils.py   # Statistical analysis tools
│   ├── MergeDB.py          # Database merging operations
│   ├── scp2xlsx.py         # Scopus to Excel converter
│   ├── wos2xlsx.py         # Web of Science to Excel converter
│   └── xlsx2vos.py         # Excel to VosViewer converter
└── WorkSpace/
    └── Project Folder/
        └── Data/            # Place your input files here
            ├── Wos txt data files
            └── Scopus csv data files
Key Files Description
Configuration Files
  • .env

    Store your API keys for Scopus and other services

  • requirements.txt

    List of required Python packages

Main Application
  • DataProcessor.py

    Entry point of the application

Conversion Tools
  • scp2xlsx.py

    Converts Scopus CSV files to Excel format

  • wos2xlsx.py

    Converts Web of Science TXT files to Excel format

  • xlsx2vos.py

    Converts Excel files to VosViewer format

Utility Modules
  • api_utils.py

    API integration and data enrichment

  • merge_utils.py

    Data merging and deduplication

  • stats_utils.py

    Statistical analysis and reporting

Follow the Workflow
  1. 1
    Create Project Folder

    Create a new folder in the Workspace directory with your project name.

    Workspace/Your_Project_Name/
  2. 2
    Prepare Data Folder

    Create a Data folder inside your project folder and place your files there:

    Workspace/Your_Project_Name/Data/
    • Scopus export files (.csv)
    • Web of Science export files (.txt)
  3. 3
    Run the Application

    Execute the program from the terminal:

    python DataProcessor.py
  4. 4
    Select and Process
    • Choose your project from the list
    • Program will automatically detect and process your files
    • Choose to enrich metadata (optional)
    • Review and confirm the processed results
  5. 5
    Results

    Find your processed files in the Analysis folder:

    Workspace/Your_Project_Name/Analysis_YYYYMMDD_HHMMSS/

Outputs

BibexPy generates the following outputs:

Unified Datasets

Combined and deduplicated data in Prefix_Bib.xlsx format.

VosViewer Files

Network visualization ready files in Prefix_Vos.txt format.

Statistics

Detailed summaries for dataset quality and completeness.

Documentation

For detailed documentation and examples, visit our comprehensive guide:

View Documentation

Support

Need Help?

For questions or feedback, contact us:

info@bibexpy.com
Report an Issue

License

BibexPy is licensed under the GNU General Public License (GPL). See the LICENSE file for details.