diff --git a/backend/docs/guide/create-feature-x.md b/backend/docs/guide/create-feature-x.md deleted file mode 100644 index 91206ebc9..000000000 --- a/backend/docs/guide/create-feature-x.md +++ /dev/null @@ -1,105 +0,0 @@ -# Guide on creating a feature in Infisical backend - -Let's say you want to implement a new feature, call it feature-x for name sake. These are steps to follow. - -## Database model change - -If there is a database change, we must first address this to generate the database schemas for us to use. - -| Create a `.env.migration` for setting the db connection uri for migration scripts or you can just export the env DB_CONNECTION_URI - -1. if you have a new table, then go to `/src/db/schemas/models.ts` and update `TableName` enum to have the new table name. -2. Then create a new migration file by running `npm run migration:new`, type the name. Keep it something related to what your about to do. For now `feature-x` -3. Now go to `/src/db/migrations/_.ts` -4. Here update both the function `up` and `down` to create/alter the postgres fields on migration up and to revert it back on migration down. [Keeping it idempotent](https://github.com/graphile/migrate/blob/main/docs/idempotent-examples.md). - -### Generate TS schemas - -Typically you would need to know write TS types for knex type sense. But we have automated this process - -1. Start the server -2. Run `npm run migration:latest` to apply all the changes to db -3. Run `npm run generate:schema`. This will generate the type and schema using [zod](https://github.com/colinhacks/zod) in `/src/db/schemas` folder. -4. Update the barrel export in `schema/index` and apply the new tables names in `/src/@types/knex.d.ts`. This will allow knex js to have typesense. - -## Business Logic - -With the database changes generated. Now let's create the APIs for `feature-x`. - -1. Run `npm run generate:component` -2. Select 1 that is service component -3. Type service name in dashcase. Like `feature-x` - -This will create a folder inside `/src/services` with `feature-x` and 3 files - -1. `feature-x-dal`: The Database Access Layer function -2. `feature-x-service`: The service layer where all business logic happens -3. `feature-x-type`: Types used by feature-x - -There are more layers like for reusable shared function u can setup a file called `feature-x-fns` - -You can use the custom infisical function `ormify` inside `src/lib/knex` to do simple db operations inside DAL. - -## Connecting the service layer with server layer - -All the server related logic happens inside `/src/server`. To connect the service layer inside server layer we use fastify plugins for dependency injection - -1. Add the service type inside `fastify.d.ts` file below `service` namespace of a FastifyServerInstance type -2. Now go to `/src/server/routes/index.ts`, instantiate the `feature-x` required dependencies like DAL layer and service layer and then pass it to `fastify.register("service,{...dependencies})` -3. With this the service layer will be accessible inside all routes under fastify service instance. It can be accessed with `server.services..` - -## Writing the routes - -1. To create a route component run `npm generate:component` -2. Select option 3 by typing it out and then type the router name in dashcase. -3. Provide the version number - -This will generate a router file inside `src/server/routes/v/` - -1. Add your logic to connect with service layer accordingly -2. Then import the router component inside the version folder index.ts. Example, If the router component was inside v1, import the the function inside `v1/index.ts` -3. Finally register it under proper prefix to access it. - -The above contains the backend folder structure. All the contribution towards backend must follow the rules - -- **scripts**: Contains all the reusable scripts used in backend automation like running migration, generating SQL schemas -- **e2e-test**: The integration test for the APIs -- **src**: Source code of backend - -## Src - -- **@types**: The type definition of some libraries like fastify, knex -- **db**: Knexjs configuration required for database. Includes migration, seed files and sql type schemas -- **lib**: Stateless reusable functions used through code base -- **queue**: Infisical queue system based on bullmq - -### Server - -- Anything related to fastify/service should be scoped inside here. -- It contains the routes, fastify plugins, server configurations -- Routes folder contains various version of routes separate into v1,v2 - -### Services - -- Core business logic for all operations -- Each service component follows co-location principle that is related things should be kept together -- Each service component contains -1. **dal**: The Database Access Layer function that contains all the db operations -2. **service**: The service layer containing all the business logic -3. **type**: The type definition used inside the service component -4. **fns**: Optional component to share reusable functions from a service related to another -5. **queue**: Optional component to put queue specific logic for a component like `secret-queue.ts` - -## EE - -- Follows same pattern as above with an exception of license change from MIT -> Infisical Proprietary License - -### Notes - -- All the services are interconnected at `/src/server/routes/index.ts`. We follow simple dependency injection principle -- All files should be in dashcases. -- Classes should not be used in codebase. Use simple functions to keep it simple -- All committed code must be linted properly by running `npm run lint:fix` and type checked using `npm run type:check` -- Try to avoid inter service shared logic as much as possible -- A controller inside a router component should try to keep it calling only one service layer. This rule could have exception when another service -like `audit-log` needs access to request object data. Then controller will call both the functions diff --git a/backend/docs/guide/folder-structure.md b/backend/docs/guide/folder-structure.md index 6b2af0487..0b76c4e23 100644 --- a/backend/docs/guide/folder-structure.md +++ b/backend/docs/guide/folder-structure.md @@ -1,7 +1,6 @@ -# Folder structure +# Backend Folder Structure Guide ``` -. ├── scripts ├── e2e-test └── src/ @@ -35,47 +34,46 @@ └── services ``` -The above contains the backend folder structure. All the contribution towards backend must follow the rules -- **scripts**: Contains all the reusable scripts used in backend automation like running migration, generating SQL schemas -- **e2e-test**: The integration test for the APIs -- **src**: Source code of backend +The following outlines the backend folder structure. All contributions to the backend should adhere to these guidelines: + +- **scripts**: This folder contains all reusable scripts for backend automation, such as running migrations and generating SQL schemas. +- **e2e-test**: Here you'll find integration tests for the APIs. +- **src**: This is the main directory for the source code of the backend. ## SRC -- **@types**: The type definition of some libraries like fastify, knex -- **db**: Knexjs configuration required for database. Includes migration, seed files and sql type schemas -- **lib**: Stateless reusable functions used through code base -- **queue**: Infisical queue system based on bullmq +- **@types**: This directory holds the type definitions for certain libraries, such as Fastify and Knex. +- **db**: In this folder, you'll find the Knex.js configuration necessary for database operations, including migration, seed files, and SQL type schemas. +- **lib**: This directory is for stateless, reusable functions used throughout the codebase. +- **queue**: This folder contains the Infisical queue system, which is based on BullMQ. ### Server -- Anything related to fastify/service should be scoped inside here. -- It contains the routes, fastify plugins, server configurations -- Routes folder contains various version of routes separate into v1,v2 +- This section is dedicated to anything related to Fastify/service and should be contained within this scope. +- It includes routes, Fastify plugins, and server configurations. +- The routes folder is organized into various versions, separated into v1, v2, etc. ### Services -- Core business logic for all operations -- Each service component follows co-location principle that is related things should be kept together -- Each service component contains - -1. **dal**: The Database Access Layer function that contains all the db operations -2. **service**: The service layer containing all the business logic -3. **type**: The type definition used inside the service component -4. **fns**: Optional component to share reusable functions from a service related to another -5. **queue**: Optional component to put queue specific logic for a component like `secret-queue.ts` +- This area handles the core business logic for all operations. +- Each service component adheres to the co-location principle, meaning related components are grouped together. +- Within each service component, you will find: + 1. **dal**: The Database Access Layer, containing all database operations. + 2. **service**: This is the service layer where all the business logic resides. + 3. **type**: Type definitions used within the service component. + 4. **fns**: An optional component for sharing reusable functions related to the service. + 5. **queue**: An optional component for queue-specific logic, such as `secret-queue.ts`. ## EE -- Follows same pattern as above with an exception of license change from MIT -> Infisical Proprietary License +- This follows the same organizational pattern as above, but with a notable change from the MIT License to the Infisical Proprietary License. ### Notes -- All the services are interconnected at `/src/server/routes/index.ts`. We follow simple dependency injection principle -- All files should be in dashcases. -- Classes should not be used in codebase. Use simple functions to keep it simple -- All committed code must be linted properly by running `npm run lint:fix` and type checked using `npm run type:check` -- Try to avoid inter service shared logic as much as possible -- A controller inside a router component should try to keep it calling only one service layer. This rule could have exception when another service - like `audit-log` needs access to request object data. Then controller will call both the functions +- All services are interconnected at `/src/server/routes/index.ts`, where we employ a straightforward dependency injection principle. +- File naming should use dash-case. +- Instead of classes, the codebase relies on simple functions to maintain simplicity. +- All code committed must be thoroughly linted using `npm run lint:fix` and type-checked with `npm run type:check`. +- Efforts should be made to minimize shared logic between services. +- Controllers within a router component should generally invoke only one service layer. Exceptions may occur, such as when a service like `audit-log` requires access to request object data, necessitating calls to multiple functions. \ No newline at end of file diff --git a/backend/docs/guide/how-to-create-a-feature.md b/backend/docs/guide/how-to-create-a-feature.md new file mode 100644 index 000000000..06a1b3610 --- /dev/null +++ b/backend/docs/guide/how-to-create-a-feature.md @@ -0,0 +1,106 @@ +# Guide to Creating a Feature in Infisical's Backend + +Suppose you're interested in implementing a new feature, let's call it "feature-x." Here are the steps you should follow: + +## Database Model Change + +If your feature involves a change in the database, you need to first address this to generate the necessary database schemas. + +- Create a `.env.migration` file to set the database connection URI for migration scripts, or alternatively, export the `DB_CONNECTION_URI` environment variable. + +1. If you're adding a new table, update the `TableName` enum in `/src/db/schemas/models.ts` to include the new table name. +2. Create a new migration file by running `npm run migration:new` and give it a relevant name, such as `feature-x`. +3. Navigate to `/src/db/migrations/_.ts`. +4. Modify both the `up` and `down` functions to create or alter Postgres fields on migration up and to revert these changes on migration down, ensuring idempotency as outlined [here](https://github.com/graphile/migrate/blob/main/docs/idempotent-examples.md). + +### Generating TS Schemas + +While typically you would need to manually write TS types for Knex type-sense, we have automated this process: + +1. Start the server. +2. Run `npm run migration:latest` to apply all database changes. +3. Execute `npm run generate:schema` to automatically generate types and schemas using [zod](https://github.com/colinhacks/zod) in the `/src/db/schemas` folder. +4. Update the barrel export in `schema/index` and include the new tables in `/src/@types/knex.d.ts` to enable type-sensing in Knex.js. + +## Business Logic + +Once the database changes are in place, it's time to create the APIs for `feature-x`: + +1. Execute `npm run generate:component`. +2. Choose option 1 for the service component. +3. Name the service in dash-case, like `feature-x`. + +This will create a `feature-x` folder in `/src/services` containing three files: + +1. `feature-x-dal`: The Database Access Layer functions. +2. `feature-x-service`: The service layer where all the business logic is handled. +3. `feature-x-type`: The types used by `feature-x`. + +For reusable shared functions, set up a file named `feature-x-fns`. + +Use the custom Infisical function `ormify` in `src/lib/knex` for simple database operations within the DAL. + +## Connecting the Service Layer to the Server Layer + +Server-related logic is handled in `/src/server`. To connect the service layer to the server layer, we use Fastify plugins for dependency injection: + +1. Add the service type in the `fastify.d.ts` file under the `service` namespace of a FastifyServerInstance type. +2. In `/src/server/routes/index.ts`, instantiate the required dependencies for `feature-x`, such as the DAL and service layers, and then pass them to `fastify.register("service,{...dependencies})`. +3. This makes the service layer accessible within all routes under the Fastify service instance, accessed via `server.services..`. + +## Writing the Routes + +1. To create a route component, run `npm generate:component`. +2. Select option 3, type the router name in dash-case, and provide the version number. + +This will generate a router file in `src/server/routes/v/`: + +1. Implement your logic to connect with the service layer as needed. +2. Import the router component in the version folder's index.ts. For instance, if it's in v1, import it in `v1/index.ts`. +3. Finally, register it under the appropriate prefix for access. + +## Backend Folder Structure + +Contributions to the backend must adhere to the following structure: + +- **scripts**: Contains reusable scripts for backend automation, like running migrations and generating SQL schemas. +- **e2e-test**: Integration tests for the APIs. +- **src**: The source code of the backend. + +### Src + +- **@types**: Type definitions for libraries like Fastify and Knex. +- **db**: Knex.js configuration for the database, including migration, seed files, and SQL type schemas. +- **lib**: Stateless, reusable functions used across the codebase. +- **queue**: Infisical's queue system based on BullMQ. + +### Server + +- Scope anything related to Fastify/service here. +- Includes routes, Fastify plugins, and server configurations. +- The routes folder contains various versions of routes separated into v1, v2, etc. + +### Services + +- Handles the core business logic for all operations. +- Follows the co-location principle: related components should be kept together. +- Each service component typically contains: + + 1. **dal**: Database Access Layer functions for database operations + 2. **service**: The service layer containing business logic. + 3. **type**: Type definitions used within the service component. + 4. **fns**: An optional component for sharing reusable functions related to the service. + 5. **queue**: An optional component for queue-specific logic, like `secret-queue.ts`. + +## EE + +- Follows the same pattern as above, with the exception of a license change from MIT to Infisical Proprietary License. + +### Notes + +- All services are interconnected at `/src/server/routes/index.ts`, following the principle of simple dependency injection. +- Files should be named in dash-case. +- Avoid using classes in the codebase; opt for simple functions instead. +- All committed code must be properly linted using `npm run lint:fix` and type-checked with `npm run type:check`. +- Minimize shared logic between services as much as possible. +- Controllers within a router component should ideally call only one service layer, with exceptions for services like `audit-log` that require access to request object data. \ No newline at end of file