Tokens in C++

Tokenà Smallest unit of a program.

 building blocks of the program's source code à C++ compiler processesà executes.

 

Six primary types of tokens in C++:

1. Keywords

Reserved words with special meaning to the compiler.

not be used as identifiers (variable names, function names, etc.).

 

list of C++ keywords:

Control Flow: if, else, switch, case, default, for, while, do, break, continue, goto, return.

Data Types: int, float, double, char, void, bool, short, long, signed, unsigned, wchar_t

Modifiers: const, volatile, mutable.

Storage Class Specifiers: auto, register, static, extern, thread_local

Type Casting: const_cast, dynamic_cast, reinterpret_cast, static_cast

Functions: inline, virtual, explicit, constexpr

Exception Handling:- try, catch, throw, noexcept

Namespaces: namespace, using

Object-Oriented Programming: class, struct, public, private, protected, friend, virtual, override, final, this

Operators: sizeof, new, delete, typeid

Templates: template, typename

Miscellaneous: true, false, nullptr, alignas, alignof, asm, static_assert, decltype

 

2. Identifiers

• Identifiers are names given to various program elements such as variables, functions, classes, objects, and arrays. Identifiers must begin with a letter or an underscore (_), followed by letters, digits, or underscores.

Rules for C++ Identifiers

1. Only Letters, Digits, and Underscores:
• include letters (A-Z, a-z), digits (0-9), and underscores (_).
• Example: myVariable, count_1.
2. Must Start with a Letter or Underscore, not start with a digit:
3. No Special Characters like @, #, $, %, etc:
4. Case-Sensitive: lower case and upper case letter not equal ex. A and a not same.
5. Not use Reserved Keyword:
6. No Spaces Allowed between variable name:

3. Constants and Literals

• Represent fixed values in a program, not computed or modified during execution.
• Integer Literals: Numbers without decimal points.
• Floating-point Literals: Numbers with decimal points.
• Character Literals: Single characters enclosed in single quotes.
• String Literals: A sequence of characters enclosed in double quotes.
• Boolean Literals: true and false.
• Null Pointer Literal: nullptr.

 C++ provides several ways to define constants, depending on the use case.

Main types are:

 1. const Keyword

used to define a constant variable.

Once initialised, its value cannot be modified.

const int MAX_USERS = 100;

const double PI = 3.14159;

 

2. constexpr Keyword

·       Defines a compile-time constant (value is computed during compilation).

constexpr int MAX_BUFFER_SIZE = 256;

constexpr double LIGHT_SPEED = 299792458.0; // Speed of light in m/s

 

·       Stricter than const and is often used for performance optimisations.

·       Functions can also be marked as constexpr to compute values at compile-time:

constexpr int square(int x) {

    return x * x;

}

constexpr int result = square(5); // Computed at compile-time

 

3. Macros (#define)

Macros are preprocessor directives for defining constants.

They are replaced by their values during preprocessing.

#define MAX_VALUE 1000

#define PI 3.14159

 

Macros are not type-safe and do not obey scope rules, so they are generally less preferred in modern C++.

 

4. Enumerations (enum)

Enums define a set of named integral constants.

enum Color { RED = 1, GREEN, BLUE };

Color favoriteColor = GREEN;

 

·       For type safety and better scoping, enum class (scoped enumerations) is preferred in modern C++:

 

enum class Direction { NORTH, SOUTH, EAST, WEST };

Direction dir = Direction::NORTH;

5. Literal Constants

Directly using values in the code,

such as:

int age = 30;         // 30 is a literal constant

double salary = 5000.0; // 5000.0 is a literal constant

 

Also specify suffixes for literals:

• L or LL for long or long long integers.
• F or f for float.
• U for unsigned integers.

long number = 123456789L;

float pi = 3.14F;

unsigned int x = 100U;

 

6. String Literals

String literals are immutable sequences of characters:

const char* greeting = "Hello, World!";

 

7. Static const

For constants that are class members, use static const to define them with class-level scope.

class Config {

public:

    static const int MAX_CONNECTIONS = 10;

};

 

8. Magic Constants

C++ provides predefined macros for special constants:

• __LINE__: Current line number in the source code.
• __FILE__: Name of the current file.
• __DATE__: Compilation date.
• __TIME__: Compilation time.

std::cout << "Compiled on " << __DATE__ << " at " << __TIME__ << '\n';

---

4. Operator

• symbols that perform operations on variables and values.A 
• wide variety of operators, including arithmetic, relational, logical, assignment, and more.

Examples:

• Arithmetic Operators: +, -, *, /, %
• Relational Operators: ==, !=, <, >, <=, >=
• Logical Operators: &&, ||, !
• Assignment Operators: =, +=, -=, *=, /=

int a = 5;

int b = 3;

int sum = a + b;  // '+' is an arithmetic operator

 

5. Punctuation (Separators)

• Used to separate different parts of a C++ program, such as statements and function parameters.
• Semicolon (;): Ends a statement.
• Comma (,): Separates parameters in a function call or variable declarations.
• Period (.): Used to access members of an object.
• Colon (:): Used in conditional operator (?:) and for defining scope (in classes).
• Parentheses (()): Used for grouping expressions, function calls, and defining function parameters.
• Braces ({ }): Used to define a block of code, like in functions or loops.
• Square Brackets ([]): Used to define arrays or access array elements.
• Angle Brackets (<>): Used for template argument lists in C++.

int arr[5];  // Square brackets used for array declaration

 

6. Comments

• Add descriptive text within the code.
• Ignored by the compiler and are meant for programmers to explain parts of the code.
• Single-line Comment: Begins with //
• Multi-line Comment: Begins with /* and ends with */

// This is a single-line comment

 

/*

  This is a multi-line comment

  that spans across multiple lines

*/

 Summary of C++ Tokens

1. Keywords: Reserved words like int, class, if, etc.
2. Identifiers: Names given to variables, functions, etc.
3. Constants/Literals: Fixed values like 5, 3.14, 'A', "Hello", etc.
4. Operators: Symbols like +, -, =, ==, etc.
5. Punctuation: Symbols like ;, {}, (), etc., used for structure.
6. Comments: Text for documentation and explanations.

========================================================