To react to community feedback we will start a feed to summarize every now and then interesting changes in this feed.
\nThis does not replace the ADR section at all.\nAs we have a lot of repositories and this will be a channel to summarize the news that are shared across the packages.\nThis will also be the way to get informed about new packages as we follow our flexible package structure.
\n", "url": "https://heptaconnect.io/news/2021-02-02-init/", "title": "Initialization of a feed", "summary": "To react to community feedback we will start a feed to summarize every now and then interesting changes in this feed", "date_modified": "2021-02-02T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2021-02-08-shopcast/", "content_html": "Edin Dedagic - host at the German podcast shopcast.fm - invited us to have a talk about HEPTACOM as part of the Shopware community and the guest Joshua Behrens.\nIt was inevitable to talk as well about HEPTAconnect.\nA lot about HEPTAconnect which even results in a split into two parts.
\nHave a deep dive into two hours of HEPTAconnect knowledge and its creators:
\n\n", "url": "https://heptaconnect.io/news/2021-02-08-shopcast/", "title": "shopcast.fm appearance", "summary": "Joshua Behrens had the chance to talk about HEPTAconnect on shopcast.fm", "date_modified": "2021-02-08T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2021-03-23-data-flow-explorer/", "content_html": "In the recent past we decided to allow explorers behave like emitters to allow easier processing of inefficient-to-read data sources.\nWe call this direct emission.\nHow to prepare your explorer for a direct emission is explained in the direct emission explorer page.\nTo show the differences in the data flow we introduce a place where you can find multiple scenarios on the new data flow overview.
\nComing soon:\nWe are currently setting up an integration guide that holds up our integration knowledge and explains our decisions that you should consider as well.\nStay tuned to let your next project experience the full data transfer expertise.
\n", "url": "https://heptaconnect.io/news/2021-03-23-data-flow-explorer/", "title": "Data flows and direct emission exploration", "summary": "Introduction to data flows and direct emission exploration", "date_modified": "2021-03-23T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2021-05-23-playground-shopware-6-4/", "content_html": "A wave of huge changes hit the playground and ironed some kinks.
\nThe major update at Shopware from 6.3 to 6.4 has been rolled out onto the HEPTAconnect packages.\nTo experience Shopware 6.4 with HEPTAconnect you can now easily grab yourself a fresh copy of the playground and dive in.\nTo allow the playground be used with your familiar ecommerce framework in a more complete manner you will now find an integrated shopware/administration.\nThis way you can use the playground to fiddle with HEPTAconnect within Shopware and in addition also with Shopware itself after importing your favorite data.
How to get your favorite data in, you might ask?\nJust use the bundled shopware platform portal.\nOn creating the shopware instance in the playground you will get a portal node for the shopware instance as well as for the artificial portal for bottles.\nRequire other portals you can already find online on GitHub under the tag heptaconnect-portal into the shopware composer.json.\nThey are automatically discovered for you to draw your first routes between the portal nodes.
\nWhat is that /repos/ folder at root level?\nWe had a look at the contribution and experimenting flow we currently provide and chose to rework that.\nThe new folder contains every HEPTAconnect package as a clone from GitHub.\nThey are linked into the vendor folder.\nHaving the different packages on this directory level makes it much easier to discover and change HEPTAconnect.\nAs they are clones from GitHub it also allows a direct way to share your work on HEPTAconnect in a bleeding edge manner.
\nHow can you know all of this? The documentation got a truck load of new content about:
\nSome of these new pages are also linked within this article so have a look around.
\n", "url": "https://heptaconnect.io/news/2021-05-23-playground-shopware-6-4/", "title": "HEPTAconnect and playground support for Shopware 6.4", "summary": "The playground supports Shopware 6.4 and the docs will tell you how to use it in just a few steps", "date_modified": "2021-05-23T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2021-05-31-queue-and-job-storage/", "content_html": "This week we completed our ADR about jobs.\nWorking on that issue really created lots of space for further tasks in the architecture of HEPTAconnect as well as on the storage layer used for the message broker.
\nIn detail, we've been using the message broker to completely keep the data in question for each transfer.\nRead our ADR about jobs and their payloads to follow our thoughts but here are the basic pros:
\nWe were testing the storage sizes on a default Shopware 6 setup.\nIt uses php serialization to store messages on the database.\nAfter our change we only store a job reference in the message and store the payload on the database.\nOur referenced payload takes up from 40% to 95% less memory on the same database driver than in the message table.\nThe following tasks benefit from the size difference:
\nAbove mentioned the architecture also draws a big benefit from that change:\nAs we now have the job payloads separated from their execution scheduling, it is easier to add reference between these jobs.\nThis is especially useful for SAT algorithms that will make their way into HEPTAconnect one day.
\n", "url": "https://heptaconnect.io/news/2021-05-31-queue-and-job-storage/", "title": "Queue and job memory utilization reduction up to 95%", "summary": "Switching to dedicated job storages reduced the memory utilization and improves job management", "date_modified": "2021-05-31T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2021-06-21-symfony-dependency-injection/", "content_html": "Write less.
\nAchieve more in less time.
\nLook at your results faster.
Read more about Dependency injection and the new default utilities.
\nOne of our main goals for HEPTAconnect is the ability to replace lots of different implementations to gain a frictionless integrability.\nWhen we were looking for the service container feature, we had an issue to solve:\nAs service container builders have not been addressed in the PSR-11, we had to implement service container and builder.\nThe custom implementation holds a 47 lines-of-code container implementation, a 36 lines-of-code container builder and a single entry point for portal developers in their code.
\nWe were able to get yet another PSR compatibility and its benefits: build and share reusable components.\nBy now our implementation of a service container is about half a year old, and we are missing features.\nWe didn't expect our small implementation to have it all right away, but it is a bit impractical in daily usage.
\nIt served the purpose, but it wasn't quite right.
\nBasically we were looking for what we were missing:
\nWe were accustomed to the Symfony and Laravel worlds having all these features.\nLaravel didn't make the cut for us as it is difficult to pull Illuminate components out of their ecosystem.\nSymfony gave us similar expectations back when we noticed Bundles are part of the HttpKernel and also quite baked into the Symfony framework /rant.
On our search for a good implementation for service containers with dependency injection we stumbled upon a benchmark.\nThe Symfony implementation was most of the time in second place but had all we ever wanted.\nSo we revisited that package and were happy to see that it had barely any dependencies which is good for an almost frictionless package.\nHaving the service definition files based on xml and yml also allows portal developers to define services without knowing the exact Symfony version behind the scenes.
\nWe gave the implementation a try and got hooked right back into it.\nEnabling all the nice things and sprinkling some more educated guesses into the container builder gave us the shiny new developer experience for portal developers we always wanted to provide:
\nWhen you already had a deeper look into our documentation, you can notice the reduction of boilerplate code in almost every portal developer code sample.\nWe added a new chapter to the documentation under portal development containing everything you need to know to use dependency injection with the new service container.\nThis way we also took the chance to provide a complete list of utility service we provide out of the box.
\n", "url": "https://heptaconnect.io/news/2021-06-21-symfony-dependency-injection/", "title": "Symfony dependency injection for portal developers", "summary": "Integrating a well known dependency injection container provides lots of cool features", "date_modified": "2021-06-21T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2021-07-12-short-notation/", "content_html": "Write less.
\nAchieve more in less time.
\nLook at your results faster.
Read more about short notation for flow components.
\nNo, the TL;DR is not a copy mistake from the previous feed entry about dependency injection.\nWe once more achieved to improve the developer experience by adding more tools to reduce your boilerplate code.\nSummarizing our ADR you can now stop writing sections of code and start to write lines of code.\nPreviously you had to write:
\n<?php\n\ndeclare(strict_types=1);\n\nnamespace FooBar\\Emitter;\n\nuse FooBar\\Packer\\BottlePacker;\nuse FooBar\\Service\\ApiClient;\nuse Heptacom\\HeptaConnect\\Dataset\\Base\\Contract\\DatasetEntityContract;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\Emission\\Contract\\EmitContextInterface;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\Emission\\Contract\\EmitterContract;\n\nclass BottleEmitter extends EmitterContract\n{\n private ApiClient $client;\n \n private BottlePacker $packer;\n \n public function __construct(ApiClient $client, BottlePacker $packer)\n { \n $this->client = $client;\n $this->packer = $packer;\n }\n\n public function run(string $externalId, EmitContextInterface $context) : ?DatasetEntityContract\n {\n return $this->packer->pack($this->client->getBottleData($externalId));\n }\n}\nSieve out the boilerplate lines and you have:
\n<?php\n\ndeclare(strict_types=1);\n\nuse FooBar\\Packer\\BottlePacker;\nuse FooBar\\Service\\ApiClient;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\Builder\\FlowComponent;\n\nFlowComponent::emitter(Bottle::class)->run(\n fn (ApiClient $client, BottlePacker $packer, string $id) => $packer->pack($client->getBottleData($id))\n);\nWhen this reminds you of Laravel routes and commands, you are right.\nThis is heavily inspired by Laravel's nature of quickly scribbling down some lines to get something big running.\nTo get an overview of the short notation and how to use it visit the documentation page.
\n", "url": "https://heptaconnect.io/news/2021-07-12-short-notation/", "title": "Short notation for flow components", "summary": "Reduce the lines of code for connecting your API to HEPTAconnect", "date_modified": "2021-07-12T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2021-07-23-easy-batch/", "content_html": "Train in step-by-step.
\nRun in batches.
Read more about the way how to batch in short notation for flow components and flow components like emitter and receiver.
\nEvery emitting or receiving flow component can improve on performance when data is processed in batches.\nIn general HEPTAconnect groups data into batches to allow the flow components to be able to act in reasonable batch sizes.\nThe documentation in the past only explained how to make a single step-by-step API interactions.\nRunning through a data transfer in such a sequential manner is due to its complexity better to outline its inner statements.\nThe flow component loops that unrolled the batches into single steps are now accessible for child classes to override.\nCompare the following real life examples how to receive manufacturer data to its successor within Shopware 6:
\n<?php\n\nuse Heptacom\\HeptaConnect\\Dataset\\Ecommerce\\Product\\Manufacturer;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\Builder\\FlowComponent;\nuse Heptacom\\HeptaConnect\\Portal\\LocalShopwarePlatform\\Support\\DalAccess;\nuse Heptacom\\HeptaConnect\\Portal\\LocalShopwarePlatform\\Unpacker;\n\nFlowComponent::receiver(Manufacturer::class)->run(static fn (\n DalAccess $dal,\n Manufacturer $manufacturer,\n Unpacker\\ManufacturerUnpacker $unpacker\n) => $dal->createSyncer()->upsert('product_manufacturer', [$unpacker->unpack($manufacturer)])->flush());\n<?php\n\nuse Heptacom\\HeptaConnect\\Dataset\\Base\\TypedDatasetEntityCollection;\nuse Heptacom\\HeptaConnect\\Dataset\\Ecommerce\\Product\\Manufacturer;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\Builder\\FlowComponent;\nuse Heptacom\\HeptaConnect\\Portal\\LocalShopwarePlatform\\Support\\DalAccess;\nuse Heptacom\\HeptaConnect\\Portal\\LocalShopwarePlatform\\Unpacker;\n\nFlowComponent::receiver(Manufacturer::class)->batch(static fn (\n DalAccess $dal,\n TypedDatasetEntityCollection $manufacturers,\n Unpacker\\ManufacturerUnpacker $unpacker\n) => $dal->createSyncer()->upsert('product_manufacturer', $manufacturers->map([$unpacker, 'unpack']))->flush());\nThere is basically no difference.\nYour project can make use of this today when you are in the lucky position to have a batch-ready API, and you've been following architectural patterns to use packer and unpacker services.
\n", "url": "https://heptaconnect.io/news/2021-07-23-easy-batch/", "title": "Easy batch just like step-by-step runs", "summary": "Convert your single runs into batch runs more easily and gain performance", "date_modified": "2021-07-23T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2021-08-10-new-documentation/", "content_html": "Maybe you already noticed it:\nWe have a beautiful new documentation website, and we think you will love it.\nOur new website is based on the material theme for MkDocs with some customizations and it's rocking a bunch of new and improved features.
\nThe HEPTAconnect documentation is now available under a new domain: heptaconnect.io.\nWe have moved from a subdomain under heptacom.de to a dedicated domain and we are also using the .io domain ending.
\nPerhaps the most notable change are the stunning visuals.\nThe main landing page shows the HEPTAconnect logo and prominently advertises the four most important sections of the documentation.\nHave you noticed the glowing backdrop behind the cards when you hover over them?
\nSpeaking of sections:\nWe have restructured our content into different sections.\nMost of our existing content went into Portal Developer and Reference.\nThe sections Integrator and Administrator are completely new and bring different perspectives into the documentation.\nTruth be told, they are not finished yet.\nBut we have big plans to extend the documentation with new content for them, so stay tuned.
\nAnd when we are already on the topic of staying tuned:\nWe noticed that most RSS readers don't update as frequently as we would like them to.\nSo starting from today we will additionally publish our blog entries on Twitter.\nSo if your RSS reader is giving you trouble, feel free to follow us there.
\nLet's get back to the star of today's show:\nThe new documentation website has an amazing new search function.\nA big problem we had with the search on the old website was the small space where the search results would get crammed into.\nSearch results were nearly indistinguishable from one another rendering them downright useless.\nThe new search bar makes use of all the space it can get to give results that are actually helpful.
\nIf you are hyped to try it out, a good first search term would be "Basic flow".\nThe first result will take you to a brand-new page with our new sequence diagrams to visualize how data flows through HEPTAconnect.\nThose diagrams show both the perspective of a portal and the perspective of HEPTAconnect core components during an exploration, an emission and a reception in a basic flow.\nLet us know if they could help you to better grasp the big picture.
\nAnd there is still more.\nMkDocs has another trick up its sleeve.\nThe entire documentation is available for download as a PDF file with a cover page and a linked table of contents.\nSo when your client demands a copy of the documentation, it's just one click away.\nTake a look on the left side of the footer.
\nLast but not least, the switch from docsify to MkDocs means that our documentation is finally visible for Google and other search engines.\nSo hopefully you can soon find your answers even faster from your favourite search engine.
\n", "url": "https://heptaconnect.io/news/2021-08-10-new-documentation/", "title": "New Documentation", "summary": "Big changes to the docs! New domain, new search, new diagrams! Read everything you need to know here.", "date_modified": "2021-08-10T00:00:00.000Z", "author": { "name": "Julian Krzefski" } }, { "id": "https://heptaconnect.io/news/2021-12-07-http-handler-flow-component/", "content_html": "In the last news the new documentation structure and engine have been celebrated.\nNow we can suggest once again to read into the new pages about our new flow component: HTTP handler.\nThis new component already has some configuration which can be managed as administrator that can be read upon in the HTTP APIs page of the administrator section.
\nWe rely fully on PSR-7 and PSR-17.\nUnfortunately PSR-15 does not fit into our just released HTTP handler structure.\nIt behaves similar to other flow components in terms of stacked processing, extensibility by portal extensions and integrations, ability to write definition in a short-notation and an object-oriented manner.
In the past you had to be an integration to host an HTTP controller to create an event driven flow over HTTP messages.\nWith HTTP handlers at hand you can now use incoming web requests and transfer them into publications or already a direct emission.\nWe already use it in the Business Central portal to build an OAuth 2 authentication flow.\nThe following example shows how to convert an incoming request into a publication of external ids about new data that is ready to pick up.
\nuse Heptacom\\HeptaConnect\\Playground\\Dataset\\Bottle;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\Builder\\FlowComponent;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\Mapping\\MappingComponentCollection;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\Mapping\\MappingComponentStruct;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\StorageKey\\Contract\\PortalNodeKeyInterface;\nuse Psr\\Http\\Message\\ResponseInterface;\nuse Psr\\Http\\Message\\ServerRequestInterface;\n\nFlowComponent::httpHandler('bottle')->post(static function (\n ServerRequestInterface $request,\n ResponseInterface $response,\n PortalNodeKeyInterface $portalNodeKey\n): ResponseInterface {\n $bottleIds = \\json_decode($request->getQueryParams()['bottle-ids']);\n\n $bottleMappings = \\array_map(static fn (string $bottleId) => new MappingComponentStruct(\n $portalNodeKey,\n Bottle::class,\n $bottleId\n ), $bottleIds);\n\n $this->publisher->publishBatch(new MappingComponentCollection($bottleMappings));\n\n return $response->withStatus(201);\n});\nDirect emission based flows can be created quickly when you already use packers to pack API data into HEPTAconnect entities.\nWith packers at hand you can use the request body, pass it into the packer and use the packed entity directly into the DirectEmissionFlow.
\n", "url": "https://heptaconnect.io/news/2021-12-07-http-handler-flow-component/", "title": "HTTP handler flow component", "summary": "HTTP handlers for webhook, OAuth flows and hosting files", "date_modified": "2021-12-07T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2022-01-11-changelogs/", "content_html": "With the release of version 0.7 we started to add changelogs.\nThese are important news and therefore reside in our news section.\nThere you can find a human-readable and easy to browse version of the changelogs.\nSo with each upcoming release of HEPTAconnect you should check out the documentation about the new changes and get excited.
\nUpdating to the most recent version of HEPTAconnect is always a combination of:
\nSo you can check your version of HEPTAconnect using $ composer show heptacom/heptaconnect-\\*, understand the news and compare.
That is super easy as we write them in a non code-exclusive focussed view to explain the changes of the public API.\nWe are focussing on feature based logs so the generic changes of a git diff turns into a human understandable view on the change.\nThe keep a changelog project thinks alike, therefore we follow its guidelines for a good changelog.\nTo summarize: Every version has a clear title with a semantic version and the release date.\nEach version has a section filled changes grouped by their classification of additions, changes, deprecations, removals, fixes and security fixes.
Looking at the changelog lines you will see they always reveal quite technical information.\nWe add them to the messages to visualize a clear connection of features to code components.\nIt makes it easy to research for unique code components and understand why they have been changed.\nWhen we do package internal changes you won't see them as they probably won't bother you.\nDon't worry though, when we change e.g. database interactions this is often considered an internal change but still resides under the changes classification for e.g. resources usage reasons.
\nYou have to take every log and look up whether it affects your code changes as every log should contain enough technical reference.\nWhen you have a match you can understand the connected feature and apply the knowledge.
\nThe process sounds correct but tedious, right?\nWe thought so as well, so we decided to snoop around one step ahead again and had a look on how to automate this, so we all have less to do again.
\nSo we like our code a bit magical but still reasonable.\nAs we do it ourselves, we are aware how people can turn PHP into anything.\nTherefore, we can't provide an automatic upgrade yet, but we are working on a tool which either supports or fixes code automatically for you.\nWe prepared guides to do upgrades for portal developers and integrators as well for contributors which we will both upgrade regularly.
\n", "url": "https://heptaconnect.io/news/2022-01-11-changelogs/", "title": "Changelogs for safer and understandable updating", "summary": "Updating your project would be a mess without proper changelogs", "date_modified": "2022-01-11T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2022-04-19-code-documentation/", "content_html": "With the release of 0.9 we started to document every interface and contract.\nThis way we ensure clear communication how to use the interface as an API consumer and as an API provider.
\nAdding documentation on source code is a delicate discussion topic in most developer conversations.\nWe evaluated our experience with other documentations and were (and still are) really happy with the approach of the PSR composer packages.\nWhen looking at e.g. the \\Psr\\Log\\LoggerInterface, there is a common description at the interface and a short hint on every method.\nIn everyday usage it was nice to use a logger instance, navigate to the interface symbol or one of the methods and see right away how the intended use is.\nA similar pattern can be seen in the documentation stubs PHPStorm ships for native PHP functions.\nThis kept the developer in the IDE without losing their focus.
We see the possibility that these comments may go out of date.\nTo prevent this from happening too often, we decided to only write expected usage and possible implementation hints.\nWith this rule of thumb we are not fixed to a certain implementation that may change over time.\nInstead, we use the intended use case as comment when the component was introduced so everyone shares the same expectation.
\n", "url": "https://heptaconnect.io/news/2022-04-19-code-documentation/", "title": "Documentation at source code level", "summary": "Providing documentation on a point that is quickly reached", "date_modified": "2022-04-19T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2022-04-26-release-0-9-0-0/", "content_html": "Two weeks ago we released the first 0.9 version.\nTo be more specific it is 0.9.0.0 and so the first version following the new version scheme described in this ADR.\nUpdated changelogs for your updating strategy can be found in the release news section.\nLet's get more detailed and hydrate the TL;DR above.
\nThe file references are a new way to transfer files with the focus on availability via HTTP and reduction of I/O operations in the HEPTAconnect installation.\nThe datasets and portals have been updated accordingly to use the new performant transport.\nSee a sample usage here.
\nFlow components are now loaded deferred and only grouped when used.\nThis allows portal extensions to provide as much flexibility as portals in terms of flow components.\nNew flexibilities are also given to portal extension developers and administrators as portal extension can now be deactivated on a stack and declared as optional.
\nWe restructured the internal management storage handling to ensure the stability and performance it deserves.\nThis includes a new package full of test scenarios against the storage base interfaces to allow also pre-built tests for any future storage provider.\nRead more here.
\nNew utilities for integrators are now available to build automatically assigned portal node configurations by patterns.\nThis enables various configurations sources like JSON files from hosting services or environment variables.\nRead more here.
\nWe provide a new service for portals to make HTTP API usage a step easier.\nThe PSR-18 compatible HttpClientContract is preconfigured to throw exceptions for 4XX and 5XX status codes, add missing headers, follow redirects and retry in case of an error or rate limit.
When a flow component is put into the log context it will be swapped out with its code origin.\nThis supports navigation to the source of an error, that is not identified by an exception.
\nAll our storage implementations start to add some form of unique identifier to the request, so it can be found more easily in the source code when found in logs.
\nEven more log messages we write and exceptions we throw have unique identifiers for quick search in the code and in the online changelog.
\nJobs have a state history that can be inspected for processing and performance analysis.
\nThe dependency version range is changed to keep everything up to date.\nThis affects doctrine/dbal, shopware/core, ramsey/uuid and dragonmantank/cron-expression as library usage has been updated to match known deprecations and raised or removed.\nRead more about its details in the changelogs.
As HEPTAconnect is looking to achieve a state-of-the-art experience on using our API, we improved our API design by ensuring to use final more frequently without removing extensibility.\nTo ensure code documentation browsing is easy, we add it to a lot of places within the code.\nEven small subtleties are taken into account like $this in short notation flow components now refers to the wrapping object-oriented flow component.
Everyone managing an HEPTAconnect instance on setup can now use --bidirectional on heptaconnect:router:add to also create a route in the reverse direction.\nMore CLI commands are supporting JSON output for easier scripting use.\nSpoiler alert: There is a huge change coming to raise the quality bar for administrator tools in the not so distant future.
Follow the news section by subscribing to the RSS feeds:
\n\nOr follow us on Twitter to get more insights on the briefly touched topics in here in the future.
\n", "url": "https://heptaconnect.io/news/2022-04-26-release-0-9-0-0/", "title": "Release 0.9.0.0", "summary": "Fast file transfers, storage stability, configuration by environment variables, improved API design", "date_modified": "2022-04-26T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2022-05-17-exception-and-log-message-codes/", "content_html": "Since the version of 0.8 we add unique codes to exceptions and log messages.\nThe full explanation can be read in this ADR.
\nIn short: these unique codes are quickly looked out for.\nEither online in our documentation or your IDE.\nThey describe a point in the source code and keeps consistent in meaning even after an update.
\nAlthough we use the unique codes for log messages and exceptions, you will likely only look them up because you are a first-responder to an issue of your system.\nSo whenever you get a code from your log message's metadata you can just look them up in the search of our documentation.\nIn general, you get a single hit within changelogs when the code has been introduced.\nThere can be more hits, when the code has been moved to a different place or removed.\nIn this scenario you can match by your installed HEPTAconnect version to find your issue.\nAs these codes are part of our public API and are in the changelogs, you will find a technical reference and a human-readable meaning to it.
\nThe real life scenario could now be something like this:
\nWhen there is the exception code 123 in your logs, and you find this message in the changelogs:
\n\n\nAdd code
\n123in\\Heptacom\\HeptaConnect\\TechnicalReference::doSomethingwhen JSON configuration file is invalid JSON.
And someone recently edited a JSON file, you can combine these hints and find a solution on your own.\nAs the changelogs are shipped with the code, you can also find this hint with a grep call in your installation.\nWhen you search the files, you will also find that single piece of code that triggered the exception right away.
This will make first-responders more helpful and independent.
\nWhen we talk about codes in log messages, we should point out that these are not correlation identifiers.\nCorrelation identifiers could only be queried in the HEPTAconnect installation as they identify a single situation like a specific web request or a job run.
\nTo keep posted with our news, that will definitely contain information on correlation ids once they are released, subscribe our feeds:
\n\nOr follow us on Twitter.
\n", "url": "https://heptaconnect.io/news/2022-05-17-exception-and-log-message-codes/", "title": "Exception and log messages in action", "summary": "First-responders looking into logging facilities will love these", "date_modified": "2022-05-17T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2022-06-07-file-references/", "content_html": "With the release of 0.9 we introduce a new feature for portal developers to vastly improve performances of file transfers.\nPreviously, transferring files always required to write them to an intermediate storage and later read them from this storage again.\nThe new FileReferences will respect the source of a file and choose the optimal transfer method accordingly.\nA portal developer can choose from three available file sources:
\nThese three strategies already cover many use cases.\nHere are some example usages for each source type:
\nuse Heptacom\\HeptaConnect\\Dataset\\Ecommerce\\Media\\Media;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\Builder\\FlowComponent;\nuse Heptacom\\HeptaConnect\\Portal\\Base\\File\\FileReferenceFactoryContract;\nuse League\\Flysystem\\Filesystem;\nuse Psr\\Http\\Message\\RequestFactoryInterface;\nuse Psr\\Http\\Message\\StreamFactoryInterface;\n\nFlowComponent::explorer(Media::class, function (\n FileReferenceFactoryContract $fileReferenceFactory\n): iterable {\n $publicUrl = 'https://picsum.photos/seed/php/300/300';\n \n $fileReference = $fileReferenceFactory->fromPublicUrl($publicUrl);\n\n $mediaEntity = new Media();\n $mediaEntity->setPrimaryKey('ad9e5b8a46364e019d9b3045b5172623');\n $mediaEntity->setFile($fileReference);\n\n yield $mediaEntity;\n});\n\nFlowComponent::explorer(Media::class, function (\n FileReferenceFactoryContract $fileReferenceFactory,\n RequestFactoryInterface $requestFactory,\n StreamFactoryInterface $streamFactory\n): iterable {\n $request = $requestFactory\n ->createRequest('POST', 'https://private-image-server.test/api/download-image/28664337703749c49af4bf198e94c396')\n ->withHeader('Authorization', 'Basic aGVwdGFjb25uZWN0OmlzIGNvb2w=')\n ->withBody($streamFactory->createStream(\\json_encode([\n 'foo' => 'bar'\n ])));\n\n $fileReference = $fileReferenceFactory->fromRequest($request);\n\n $mediaEntity = new Media();\n $mediaEntity->setPrimaryKey('ad9e5b8a46364e019d9b3045b5172623');\n $mediaEntity->setFile($fileReference);\n\n yield $mediaEntity;\n});\n\nFlowComponent::explorer(Media::class, function (\n FileReferenceFactoryContract $fileReferenceFactory,\n Filesystem $filesystem\n): iterable {\n $fileContents = $filesystem->read('data/images/a2ef9e0ef17e4b3585041ea36c4cde13/1.jpg');\n\n $fileReference = $fileReferenceFactory->fromContents($fileContents);\n\n $mediaEntity = new Media();\n $mediaEntity->setPrimaryKey('ad9e5b8a46364e019d9b3045b5172623');\n $mediaEntity->setFile($fileReference);\n\n yield $mediaEntity;\n});\nWe recommend to avoid using the source type "File contents", because it leads to I/O operations and preventing those will greatly improve performance.\nIn an effort to prevent usages of "File contents" and therefore improve performance, you can utilize HTTP handlers to tunnel file downloads through HEPTAconnect.\nThe idea is to generate a URL to be used as "Public URL" file source.\nThis URL points to an HTTP handler that will perform the file download and return its contents as its response body.\nTake a look at the new article Send files from an FTP server to learn more about this topic and see our example code.
\n", "url": "https://heptaconnect.io/news/2022-06-07-file-references/", "title": "File reference for direct and fast file transfer", "summary": "Tunnel file transfer and make best use of external APIs", "date_modified": "2022-06-07T00:00:00.000Z", "author": { "name": "Julian Krzefski" } }, { "id": "https://heptaconnect.io/news/2022-11-29-production-template/", "content_html": "HEPTAconnect is (at its core) completely framework-agnostic.\nThat means, it can virtually be integrated into any kind of PHP-based application.\nThis approach makes HEPTAconnect highly versatile, but it also requires integrating HEPTAconnect into an application to provide a runtime environment.
\nWe aim to make setting up HEPTAconnect projects as simple as possible, so we created a template repository to kickstart your projects.\nThis repository is designed to be forked and then adjusted to your requirements.\nHere you can check out our HEPTAconnect production template repository on GitHub and fork it yourself.\nYou can also use composer create-project to start your new project.
composer create-project heptaconnect/production my-project -s dev\nAt this point you should take a look at the README.md file.\nThere you will find a quick introduction and useful first steps.
This repository template is our recommendation to create new projects with HEPTAconnect.\nWe will continue to maintain it to stay up to date with the latest HEPTAconnect releases.\nWe also use this repository internally to kickstart our own HEPTAconnect integrations.
\nCurrently, the production template depends on shopware/core for some dev-ops utilities and uses it as runtime environment for HEPTAconnect.\nWhile this is not really a problem, we aim to be fully independent from shopware/core to trim down on our required composer packages.\nWe have that planned for version 0.10 of HEPTAconnect.
Furthermore, we plan to release more documentation content about the production template.\nAs of now, the README.md file inside the repository is the only documentation for it.\nMore detailed documentation that covers more use cases will follow soon™ in the Integrator section on the HEPTAconnect documentation website.
With HTTP handlers we already solved different challenges in the past:
\nSo many use cases, where we could use an abstraction layer to share common code efficiently.\nGood news: PSR-15 solves this problem conceptually.
With 0.9.2 we added support for PSR-15 middlewares for HTTP handlers.\nThis means, you can add middlewares to your portal, which are automatically applied to all inbound HTTP requests and responses.\nUsing this very powerful concept allows you to add custom logic like authorization, logging, caching, etc. to your portal.
\nPSR-15 is unfortunately not made for HTTP clients.\nThe PSR-7 HTTP client can already have shared common code with the use of decorators, which are very similar to PSR-15 middlewares.\nUsing these decorators is a good and known strategy but it is not as easy to use as the automatically applied PSR-15 middlewares.\nTherefore, we created our own middleware interface for HTTP clients, that is similar to PSR-15 middleware specification.
\nThese two middleware interfaces will support you to build your own helpful tools for your project.\nHave a look at the documentation pages for the PSR-15 middleware interface and the HTTP client middleware interface to learn more about the interfaces and how to use them.
\n", "url": "https://heptaconnect.io/news/2023-02-14-http-middleware/", "title": "HTTP middlewares for clients and servers", "summary": "Interception points for HTTP requests and responses in every direction", "date_modified": "2023-02-14T00:00:00.000Z", "author": { "name": "Joshua Behrens" } }, { "id": "https://heptaconnect.io/news/2023-08-15-code-execution-in-repl/", "content_html": "composer require psysh you can use heptaconnect:repl, which integrates psysh into HEPTAconnectREPL stands for Read-Eval-Print-Loop.\nIt is a common pattern in programming languages to provide a way to execute code without prior writing and deployment.\nYou are likely familiar with this pattern from the browser console or the php -a/php --interactive command.\nThe php -a command is a REPL for PHP, that is shipped with the PHP interpreter itself.\nTry it out yourself by simply typing php -a into your terminal and execute a command like var_dump(new DateTime()).\nIt will print the current date and time.\nThe execution did not stop though.\nYou can execute multiple commands in a row now by entering a new command.\nThis is where the loop comes in.\nIt will keep executing commands until you exit the REPL by typing exit or pressing Ctrl+D.
The php interactive mode is a very simple REPL, that does not provide any additional features you are used to from IDEs or other development tools.\npsysh is a REPL for PHP written in PHP itself and shipped as a standalone package, that can be used as a replacement for php --interactive.\nIn comparison, it is a very powerful REPL, that provides a lot of features, that you know from your IDE.\nYou can look up code behind classes, read documentation, autocomplete code and much more.\nIt is really versatile and that is why we integrated it into HEPTAconnect.
Once you composer require psysh you can use the heptaconnect:repl command, which integrates psysh into HEPTAconnect.\nIt will automatically load a portal node container of your choice and provide you with a REPL, that is already configured to work with that specific portal node.
As example you can paste the following code into the REPL and execute it:
\nservice(\\Psr\\Log\\LoggerInterface::class)->info('Hello World');\nThis will log the message Hello World to the portal node specific logger.\nYou can access any service out of the portal node container and execute any code you want.